Advanced Usage

This page includes details about some advanced features that Intel Owl provides which can be optionally enabled. Namely,

Organizations and User management

Starting from ThreatMatrix v4, a new "Organization" section is available on the GUI. This section substitute the previous permission management via Django Admin and aims to provide an easier way to manage users and visibility.

Multi Tenancy

Thanks to the "Organization" feature, ThreatMatrix can be used by multiple SOCs, companies, etc...very easily. Right now it works very simply: only users in the same organization can see analysis of one another. An user can belong to an organization only.

Manage organizations

You can create a new organization by going to the "Organization" section, available under the Dropdown menu you cand find under the username.

Once you create an organization, you are the unique "Owner" of that organization. So you are the only one who can delete the organization and promote/demote/kick users. Another role, which is called "Admin", can be set to a user (via the Django Admin interface only for now). Owners and admins share the following powers: they can manage invitations and the organization's plugin configuration.

Accept Invites

Once an invite has sent, the invited user has to login, go to the "Organization" section and accept the invite there. Afterwards the Administrator will be able to see the user in his "Organization" section.

Plugins Params and Secrets

From ThreatMatrix v4.1.0, Plugin Parameters and Secrets can be defined at the organization level, in the dedicated section. This allows to share configurations between users of the same org while allowing complete multi-tenancy of the application. Only Owners and Admins of the organization can set, change and delete them.

Disable Plugins at Org level

The org admin can disable a specific plugin for all the users in a specific org. To do that, Org Admins needs to go in the "Plugins" section and click the button "Enabled for organization" of the plugin that they want to disable.

Registration

Since ThreatMatrix v4.2.0 we added a Registration Page that can be used to manage Registration requests when providing ThreatMatrix as a Service.

After a user registration has been made, an email is sent to the user to verify their email address. If necessary, there are buttons on the login page to resend the verification email and to reset the password.

Once the user has verified their email, they would be manually vetted before being allowed to use the ThreatMatrix platform. The registration requests would be handled in the Django Admin page by admins. If you have ThreatMatrix deployed on an AWS instance with an IAM role you can use the SES service.

To have the "Registration" page to work correctly, you must configure some variables before starting ThreatMatrix. See Optional Environment Configuration

In a development environment the emails that would be sent are written to the standard output.

Optional Analyzers

Some analyzers which run in their own Docker containers are kept disabled by default. They are disabled by default to prevent accidentally starting too many containers and making your computer unresponsive.

Name	Analyzers	Description
Malware Tools Analyzers	`PEframe_Scan` `Capa_Info` `Floss` `Strings_Info` `ClamAV` `APKiD` `Thug_URL_Info`, `Thug_HTML_Info` `BoxJS` `Qiling_Windows`, `Qiling_Windows_Shellcode`, `Qiling_Linux`, `Qiling_Linux_Shellcode`	PEFrame performs static analysis on Portable Executable malware and malicious MS Office documents Capa detects capabilities in executable files FLOSS automatically deobfuscate strings from malware binaries String_Info_Classic extracts human-readable strings where as ML version of it ranks them ClamAV antivirus engine scans files for trojans, viruses, malwares using a multi-threaded daemon APKiD identifies many compilers, packers, obfuscators, and other weird stuff from an APK or DEX file. Thug performs hybrid dynamic/static analysis on a URL or HTML page. Box-JS is a tool for studying JavaScript malware Qiling is a tool for emulating the execution of a binary file or a shellcode. It requires the configuration of its rootfs, and the optional configuration of profiles. The rootfs can be copied from the Qiling project: please remember that Windows dll must be manually added for license reasons. Qiling provides a DllCollector to retrieve dlls from your licensed Windows. Profiles must be placed in the `profiles` subfolder
TOR Analyzers	`Onionscan`	Scans TOR .onion domains for privacy leaks and information disclosures.
CyberChef	`CyberChef`	Run a transformation on a CyberChef server using pre-defined or custom recipes(rules that describe how the input has to be transformed). Check further instructions here
PCAP Analyzers	`Suricata`	You can upload a PCAP to have it analyzed by Suricata with the open Ruleset. The result will provide a list of the triggered signatures plus a more detailed report with all the raw data generated by Suricata. You can also add your own rules (See paragraph "Analyzers with special configuration"). The installation is optimized for scaling so the execution time is really fast.
PhoneInfoga	`PhoneInfoga_scan`	PhoneInfoga is one of the most advanced tools to scan international phone numbers. It allows you to first gather basic information such as country, area, carrier and line type, then use various techniques to try to find the VoIP provider or identify the owner. It works with a collection of scanners that must be configured in order for the tool to be effective. PhoneInfoga doesn't automate everything, it's just there to help investigating on phone numbers. here

To enable all the optional analyzers you can add the option --all_analyzers when starting the project. Example:

./start prod up --all_analyzers

Otherwise you can enable just one of the cited integration by using the related option. Example:

./start prod up --tor_analyzers

Customize analyzer execution

Some analyzers provide the chance to customize the performed analysis based on parameters that are different for each analyzer.

from the GUI

You can click on "Runtime Configuration" button in the "Scan" page and add the runtime configuration in the form of a dictionary. Example:

"VirusTotal_v3_File": {
    "force_active_scan_if_old": true
}

from Pythreatmatrix

While using send_observable_analysis_request or send_file_analysis_request endpoints, you can pass the parameter runtime_configuration with the optional values. Example:

runtime_configuration = {
    "Doc_Info": {
        "additional_passwords_to_check": ["passwd", "2020"]
    }
}
pythreatmatrix_client.send_file_analysis_request(..., runtime_configuration=runtime_configuration)

PhoneInfoga

PhoneInfoga provides several Scanners to extract as much information as possible from a given phone number. Those scanners may require authentication, so they're automatically skipped when no authentication credentials are found.

By default the scanner used is local. Go through this guide to initiate other required API keys related to this analyzer.

CyberChef

You can either use pre-defined recipes or create your own as explained here.

To use a pre-defined recipe, set the predefined_recipe_name argument to the name of the recipe as defined here. Else, leave the predefined_recipe_name argument empty and set the custom_recipe argument to the contents of the recipe you want to use.

Additionally, you can also (optionally) set the output_type argument.

Pre-defined recipes

"to decimal": [{"op": "To Decimal", "args": ["Space", False]}]

Analyzers with special configuration

Some analyzers could require a special configuration:

GoogleWebRisk: this analyzer needs a service account key with the Google Cloud credentials to work properly. You should follow the official guide for creating the key. Then you can populate the secret service_account_json for that analyzer with the JSON of the service account file.

ClamAV: this Docker-based analyzer uses clamd daemon as its scanner and is communicating with clamdscan utility to scan files. The daemon requires 2 different configuration files: clamd.conf(daemon's config) and freshclam.conf (virus database updater's config). These files are mounted as docker volumes in /integrations/malware_tools_analyzers/clamav and hence, can be edited by the user as per needs, without restarting the application. Moreover ClamAV is integrated with unofficial open source signatures extracted with Fangfrisch. The configuration file fangfrisch.conf is mounted in the same directory and can be customized on your wish. For instance, you should change it if you want to integrate open source signatures from SecuriteInfo

Suricata: you can customize the behavior of Suricata:
- /integrations/pcap_analyzers/config/suricata/rules: here there are Suricata rules. You can change the custom.rules files to add your own rules at any time. Once you made this change, you need to either restart ThreatMatrix or (this is faster) run a new analysis with the Suricata analyzer and set the parameter reload_rules to true.
- /integrations/pcap_analyzers/config/suricata/etc: here there are Suricata configuration files. Change it based on your wish. Restart ThreatMatrix to see the changes applied.

Yara:
- You can customize both the repositories parameter and private_repositories secret to download and use different rules from the default that ThreatMatrix currently support.
  - The repositories values is what will be used to actually run the analysis: if you have added private repositories, remember to add the url in repositories too!
- You can add local rules inside the directory at /opt/deploy/files_required/yara/YOUR_USERNAME/custom_rules/. Please remember that these rules are not synced in a cluster deploy: for this reason is advised to upload them on GitHub and use the repositories or private_repositories attributes.

Notifications

Since v4, ThreatMatrix integrated the notification system from the certego_saas package, allowing the admins to create notification that every user will be able to see.

The user would find the Notifications button on the top right of the page:

There the user can read notifications provided by either the administrators or the ThreatMatrix Maintainers.

As an Admin, if you want to add a notification to have it sent to all the users, you have to login to the Django Admin interface, go to the "Notifications" section and add it there. While adding a new notification, in the body section it is possible to even use HTML syntax, allowing to embed images, links, etc; in the app_name field, please remember to use threatmatrix as the app name.

Everytime a new release is installed, once the backend goes up it will automatically create a new notification, having as content the latest changes described in the CHANGELOG.md, allowing the users to keep track of the changes inside threatmatrix itself.