Many forensic artifacts can be looked at from many different angles. A few years ago I proposed a concept of filighting that tried to solve a problem of finding unusual, orphaned and potentially malicious files dropped inside directories that contain files that DO NOT reference these orphaned files at all.

I really hope that forensic analysis tools will evolve to add more features that will help to automate file system analysis based not only on a list of known hashes and/or file extensions, but also paths, partial (relative) paths, file names, actual file types based on their content, and ideas that rely on more complex algorithms: using prebuilt artifacts collections, leveraging various correlations (ideas like filighting), and of course machine learning and AI.

Today I want to explore one more angle of looking at file system artifacts — classes of file content. There are many file formats out there: executables, documents, configuration files, database files, and many other file types. The classification I am focusing on today though is slightly different – the format itself doesn’t interest me too much, but the function of the file does…

My guinea pig will be a license file. The type of a file that is all over the place, but no one reads them. And yes, removing them from the examiner’s view (during file system analysis) may not add a lot of value, but it’s used here only to illustrate the idea. There are many other file classes like this that can be classified as noise to the examiners’ eyes and if we start clustering them together, who knows, maybe we have just saved some personhours there…

I asked myself the following question:

– having a file system in front of me, how do I find all license files on it?

There are at least a few approaches I can think of:

• use hashes of known license files,
• use file names typically used by license files,
• analyze content of all files and look for content that resembles a license file.

All of them have their own challenges:

• the first one needs a lot of prep work to collect good hashes,
• the second one is hard to do w/o some proper analysis of a clean sampleset, and
• the third one is the most reliable, but it’s slow & needs even more preparation because it has to take into account a few more aspects: localization issues (license in various languages), file encoding issues (Unicode variants, ASCII, MBCS), file formats (TXT, RTF, HTM(L), PDF, DOC(X), etc.), and of course — performance (reading many files to analyze their content is expensive, plus not every file referencing GPL, LGPL, GNU is a license file)

I am going to focus here on the second one.

Your typical license file is usually called license, license.txt, eula.txt, and in case of Open Source, we often see files named like gpl.txt, license.gpl.txt, lgpl.txt, etc.

When you start researching this file naming bit a bit more, you will soon realize that there are a lot of variations. A lot of issues listed in 3rd point come to play as well f.ex.:

• file names can be localized,
• file extensions can be .txt, .rtf, .htm(l), .doc(x), .pdf, .xml,
• some of the file names have typos,
• many license file names use various prefixes or suffixes that identify the licensed software, the language or code page identifying the language the license file is written in,
• some file names may refer to compressed file names f.ex. *.tx_ (in installation packages),
• some license files may be stored inside the archives (including password-protected files) or installers,
• some licenses are embedded inside the compiled help files (.hlp, .chm),
• some programs may be hiding the licensing information in files named with various infixes: copying, releasenotes, thirdparty, copyright, and their variants, etc.,
• some may refer to software version in terms of full, trial f.ex. evaluation,
• some files with a license in name often refer to actual software licensing (getting keys, subscription, transferring the licenses, etc.),
• finally, some file names may be available in a 8.3 DOS notation only.

As usual, the more you look, the more complex the problem you see.

For this post I have compiled a large file containing possible license file names. You can download it here.

Will it make anybody’s life easier?

I don’t know.

What matters is that we learned a little bit more how difficult the process of automated file system analysis is. What started as a trivial and frivolous idea ended up being a Don Quixotish attempt to formalize something that is impossible to tackle, even with a data-heavy approach…

I have mapped an extensive list of Chrome Plug-in IDs to their names before. Of course, I knew for a long time that I will need to take a look at Firefox Add-ons too…. And in fairness, I did… I actually made a few approaches in the past but without much success.

If you are not familiar with Firefox Add-ons IDs, here’s your starter pack.

The forensics ramifications of the choices described in that article cannot be overlooked:

• Add-on IDs are not MANDATORY – yup, some Add-ons use (unique) IDs, some – don’t
• There are at least 2 versions of Add-on JSON manifest to take care of (manifest being the manifest.json file storing all the info about the plug-in, or so you wish — read below)

What the article doesn’t tell you is that the Firefox Add-on world is a total mess.

What that means is that you can’t just download all Firefox Add-ons in one go and then parse their manifests to build a table with all the names, IDs, and their descriptions…

Oh, no… this would be too easy.

The moment you try you will immediately discover that:

• many manifest.json files are not following JSON standard requirements and can’t be parsed with standard Python libraries (f.ex. comments are not allowed and generate exceptions; hint: json5 works)
• many entries inside these files include Unicode characters or encoded Unicode characters that can’t be rendered properly with standard Python libraries
• many newer manifest.json files reference external _locales\<language-code> folders storing one or multiple messages.json files defining referenced localized strings inside the manifest.json files; while these external references are easy to spot as they are all prefixed with __MSG_ it’s an additional complexity
• It’s all nice and cozy, except:
  • list of locale language-specific directories is messy and non-standard: en, en-us, en_us, en_gb – this is just for US and UK English and it’s just not quality-checked at all
  • many messages.json files do not include all localized fields (yup, your en\messages.json may define 2 strings, and ru\messages.json 20!)
  • many messages.json files include typos
  • many messages.json files include field names referenced by manifest.json files that are stored in a way that makes them inaccessible f.ex. fully capitalized string inside manifest.json file and lower-case, or camel-case inside the messages.json – seriously, is no one checking this when add-ons are submitted? :shockemoji:

I wish this was over, but it is not. So, I don’t have a list of Firefox Add-on ID to name mapping for you yet. I only have this rant 🙂 Sorry 🙂