Finding Alternate Data Streams (ADS) with HMFT

Finding Alternate Data Streams  (ADS) on the whole drive may be quite time consuming so in this quick post I will show you how to do it faster with HMFT.

As you probably know, the latest version of HMFT supports listing of basic attributes directly from $MFT – from both images and live systems. Amongst the features it currently supports is showing type of attribute and its name. Turns out, that this is enough information to find out what named DATA streams are hidden inside the FILE records – and this is essentially what ADSs are.

So…

First, let’s test how HMFT shows ADS-related data:

  • First let’s create a few sample ADSs
echo > f:\test
echo > f:\test:ads
echo > f:\test:ads2
echo > f:\test:ads3
  • Next, we run hmft over the drive and saving it to a file
hmft -l f: f_mft.txt
  • Finally, let’s see the content of the file – scroll down to see file name, first unnamed DATA attribute that is then followed by 3 named DATA attributes – ADS names:
  [FILE]
    SignatureD                    = 1162627398
    OffsetToFixupArrayW           = 48
    NumberOfEntriesInFixupArrayW  = 3
    LogFileSequenceNumberQ        = 4204637
    SequenceValueW                = 1
    LinkCountW                    = 1
    OffsetToFirstAttributeW       = 56
    FlagsW                        = 1
    UsedSizeOfMFTEntryD           = 448
    AllocatedSizeOfMFTEntryD      = 1024
    FileReferenceToBaseRecordQ    = 0
    NextAttributeIdD              = 6
   --

    RESIDENT ATTRIBUTE
      AttributeTypeIdentifierD = 16
      LengthOfAttributeD       = 96
      NonResidentFlagB         = 0
      LengthOfNameB            = 0
      OffsetToNameW            = 0
      FlagsW                   = 0
      AttributeIdentifierW     = 0
      --
      SizeOfContentD          = 72
      OffsetToContentW        = 24
      --
        MFTA_STANDARD_INFORMATION
            CreationTimeQ         = 129938289425003390
            ModificationTimeQ     = 129938289502223390
            MFTModificationTimeQ  = 129938289502223390
            AccessTimeQ           = 129938289425003390
            FlagsD                = 32
            MaxNumOfVersionsD     = 0
            VersionNumberD        = 0
            ClassIdD              = 0
            OwnerIdD              = 0
            SecurityIdD           = 261
            QuotaQ                = 0
            USNQ                  = 0
            CreationTime (epoch)    = 1349355342
            ModificationTime (epoch)  = 1349355350
            MFTModificationTime (epoch)  = 1349355350
            AccessTime (epoch)           = 1349355342
   --

    RESIDENT ATTRIBUTE
      AttributeTypeIdentifierD = 48
      LengthOfAttributeD       = 104
      NonResidentFlagB         = 0
      LengthOfNameB            = 0
      OffsetToNameW            = 0
      FlagsW                   = 0
      AttributeIdentifierW     = 2
      --
      SizeOfContentD          = 74
      OffsetToContentW        = 24
      --
        MFTA_FILE_NAME
            ParentID6             = 5
            ParentUseIndexW       = 5
            CreationTimeQ         = 129938289425003390
            ModificationTimeQ     = 129938289425003390
            MFTModificationTimeQ  = 129938289425003390
            AccessTimeQ           = 129938289425003390
            CreationTime (epoch)    = 1349355342
            ModificationTime (epoch)  = 1349355342
            MFTModificationTime (epoch)  = 1349355342
            AccessTime (epoch)           = 1349355342
            AllocatedSizeQ        = 0
            RealSizeQ             = 0
            FlagsD                = 32
            ReparseValueD         = 0
            LengthOfNameB         = 4
            NameSpaceB            = 3
     FileName = test
   --

    RESIDENT ATTRIBUTE
      AttributeTypeIdentifierD = 128
      LengthOfAttributeD       = 40
      NonResidentFlagB         = 0
      LengthOfNameB            = 0
      OffsetToNameW            = 24
      FlagsW                   = 0
      AttributeIdentifierW     = 1
      --
      SizeOfContentD          = 13
      OffsetToContentW        = 24
      --
        MFTA_DATA
   --

    RESIDENT ATTRIBUTE
      AttributeTypeIdentifierD = 128
      LengthOfAttributeD       = 48
      NonResidentFlagB         = 0
      LengthOfNameB            = 3
      OffsetToNameW            = 24
      FlagsW                   = 0
      AttributeIdentifierW     = 3
      --
      SizeOfContentD          = 13
      OffsetToContentW        = 32
      --
        MFTA_DATA
    AttributeName = ads
   --

    RESIDENT ATTRIBUTE
      AttributeTypeIdentifierD = 128
      LengthOfAttributeD       = 48
      NonResidentFlagB         = 0
      LengthOfNameB            = 4
      OffsetToNameW            = 24
      FlagsW                   = 0
      AttributeIdentifierW     = 4
      --
      SizeOfContentD          = 13
      OffsetToContentW        = 32
      --
        MFTA_DATA
    AttributeName = ads2
   --

    RESIDENT ATTRIBUTE
      AttributeTypeIdentifierD = 128
      LengthOfAttributeD       = 48
      NonResidentFlagB         = 0
      LengthOfNameB            = 4
      OffsetToNameW            = 24
      FlagsW                   = 0
      AttributeIdentifierW     = 5
      --
      SizeOfContentD          = 13
      OffsetToContentW        = 32
      --
        MFTA_DATA
    AttributeName = ads3

 

  • Knowing all this, we can quickly put together a perl script that can walk through the data and pick up all ADS from the output file:
use strict;
my $f='';
my $l='';
while (<>)
{
  s/[\r\n]+//g;
  $f = $1 if /FileName = (.+)$/;
  print "$f:$1\n" if ($l =~ /MFTA_DATA/&&/AttributeName = (.+)$/);
  $l = $_;
}
  • Save it as ads.pl
  • Run it using the following syntax
perl ads.pl <hmft output>

e.g.:

perl ads.pl f_mft.txt

The output for the example file system is:

$Repair:$Config
test:ads
test:ads2
test:ads3

I suggest you running a test on your local drives  – you are probably going to be quite surprised 🙂

Not only you may find plenty of files with ADS, but you may also get to know less-known good ADSs – many of them I have listed previously and a few more e.g. internal ADSs used by OS:

  • $Info in $UpCase:$Info
  • $Config in $Repair:$Config
  • $Max in $UsnJrnl:$Max

and also MAC-related streams (resource forks) added by Safari  (kinda equivalents of IE’s Zone.Identifier)

  • com.apple.quarantine
  • com.apple.metadata:kMDItemWhereFroms

Note on a small bug here: with a larger number of ADSs the ads.pl script will show incorrect entries as ADS attributes that don’t fit within one FILE record will be stored elsewhere and w/o FILENAME attribute, hence the associated file name will be incorrect. Some may be also stored under ATTRIBUTE_LIST that is not supported by HMFT yet.

HMFT update: listing $MFT attributes

A few months back I released the first version of HMFT – a small utility written in x86 assembly that reads $MFT directly from a physical disk (or raw image file/DD format) and saves it to a file. Today I am releasing a new version of this tool that now can also extract $MFT metadata and print it out to the output file. It is very similar to AnalyzeMFT from David Kovar, mft.pl (wfa3e.zip) from Harlan Carvey, and fls from Sleuthkit as well as other similar utilities.

The main difference is that it is very small, fast, works on both live systems and images, and tries to parse the attributes and print out raw data in a way that includes all gore details from $MFT FILE records to help in analysis and  learning the NTFS internals.

Apart from a new functionality, I also fixed one bug – the actual $MFT FILE record was not saved to the output file in a previous version; this is now fixed.

As usual:

  • it’s a work in progress and at the moment it only supports FILE_NAME and STANDARD_INFORMATION attributes as well as data LCNs. Hopefully I will be able to add other information later on.
  • it may contain bugs so if you spot any, please do let me know and I will try to fix them.
  • any feedback is much appreciated, thanks!

Download a new version here.

Enjoy!

The new version now takes 3 arguments from a command line:

Usage:
   hmft [drive:] [-/options] [output filename]
      where options are:
      - l - enumerate $MFT and list FILE record attributes (partially implemented)
      - d - dump $MFT to a file

Examples:
   hmft -d c: c_mft.dat
   hmft -l c: c_mft_listing.dat

Example session on a 1.2GiB $MFT:

Example output:

[NTFS BOOT RECORD]
  BytesPerSector = 512
  SectorsPerCluster = 8
  MFTStartCluster = 786432
  ----------------------------------------------
  [FILE]
    SignatureD                    = 1162627398
    OffsetToFixupArrayW           = 48
    NumberOfEntriesInFixupArrayW  = 3
    LogFileSequenceNumberQ        = 99422051935
    SequenceValueW                = 1
    LinkCountW                    = 1
    OffsetToFirstAttributeW       = 56
    FlagsW                        = 1
    UsedSizeOfMFTEntryD           = 616
    AllocatedSizeOfMFTEntryD      = 1024
    FileReferenceToBaseRecordQ    = 0
    NextAttributeIdD              = 7
   --

    RESIDENT ATTRIBUTE
      AttributeTypeIdentifierD = 16
      LengthOfAttributeD       = 96
      NonResidentFlagB         = 0
      LengthOfNameB            = 0
      OffsetToNameW            = 24
      FlagsW                   = 0
      AttributeIdentifierW     = 0
      --
      SizeOfContentD          = 72
      OffsetToContentW        = 24
      --
        MFTA_STANDARD_INFORMATION
            CreationTimeQ         = 128880037529117193
            ModificationTimeQ     = 128880037529117193
            MFTModificationTimeQ  = 128880037529117193
            AccessTimeQ           = 128880037529117193
            FlagsD                = 6
            MaxNumOfVersionsD     = 0
            VersionNumberD        = 0
            ClassIdD              = 0
            OwnerIdD              = 0
            SecurityIdD           = 256
            QuotaQ                = 0
            USNQ                  = 0
            CreationTime (epoch)    = 1243530152
            ModificationTime (epoch)  = 1243530152
            MFTModificationTime (epoch)  = 1243530152
            AccessTime (epoch)           = 1243530152
   --

    RESIDENT ATTRIBUTE
      AttributeTypeIdentifierD = 48
      LengthOfAttributeD       = 104
      NonResidentFlagB         = 0
      LengthOfNameB            = 0
      OffsetToNameW            = 24
      FlagsW                   = 0
      AttributeIdentifierW     = 3
      --
      SizeOfContentD          = 74
      OffsetToContentW        = 24
      --
        MFTA_FILE_NAME
            ParentID6             = 5
            ParentUseIndexW       = 5
            CreationTimeQ         = 128880037529117193
            ModificationTimeQ     = 128880037529117193
            MFTModificationTimeQ  = 128880037529117193
            AccessTimeQ           = 128880037529117193
            CreationTime (epoch)    = 1243530152
            ModificationTime (epoch)  = 1243530152
            MFTModificationTime (epoch)  = 1243530152
            AccessTime (epoch)           = 1243530152
            AllocatedSizeQ        = 1051983872
            RealSizeQ             = 1051983872
            FlagsD                = 6
            ReparseValueD         = 0
            LengthOfNameB         = 4
            NameSpaceB            = 3
     FileName = $MFT
   --

    NON_RESIDENT ATTRIBUTE
      AttributeTypeIdentifierD = 128
      LengthOfAttributeD       = 80
      NonResidentFlagB         = 1
      LengthOfNameB            = 0
      OffsetToNameW            = 64
      FlagsW                   = 0
      AttributeIdentifierW     = 1
      --
      StartingVCNQ          = 0
      EndingVCNQ            = 293647
      OfsToRunListW         = 64
      CompressionUnitSizeW  = 0
      UnusedD               = 0
      AllocateSizeQ         = 1202782208
      ActualSizeQ           = 1202782208
      InitializedSizeQ      = 1202782208
      --
        MFTA_DATA
              len = 2
              ofs = 3
              LCN_Ofs = 786432
              LCN_Len = 17312
              len = 3
              ofs = 4
              LCN_Ofs = 16909768
              LCN_Len = 276336
              len = 0
              ofs = 0
   --

    NON_RESIDENT ATTRIBUTE
      AttributeTypeIdentifierD = 176
      LengthOfAttributeD       = 272
      NonResidentFlagB         = 1
      LengthOfNameB            = 0
      OffsetToNameW            = 64
      FlagsW                   = 0
      AttributeIdentifierW     = 6
      --
      StartingVCNQ          = 0
      EndingVCNQ            = 36
      OfsToRunListW         = 64
      CompressionUnitSizeW  = 0
      UnusedD               = 0
      AllocateSizeQ         = 151552
      ActualSizeQ           = 148896
      InitializedSizeQ      = 148896
      --
        MFTA_BITMAP
  NumOfClustersBlocks = 2
  ----------------------------------------------

Download a new version here.