1
mirror of https://github.com/mvt-project/mvt synced 2025-10-21 22:42:15 +02:00

Compare commits

...

16 Commits

13 changed files with 195 additions and 99 deletions

8
docs/android/adb.md Normal file
View File

@@ -0,0 +1,8 @@
# Check over ADB
TODO
<!-- In order to use `mvt-android` you need to connect your Android device to your computer. You will then need to [enable USB debugging](https://developer.android.com/studio/debug/dev-options#enable>) on the Android device.
If this is the first time you connect to this device, you will need to approve the authentication keys through a prompt that will appear on your Android device.
-->

View File

@@ -1,36 +1,45 @@
# Checking SMSs from Android backup
# Check an Android Backup (SMS messages)
Some attacks against Android phones are done by sending malicious links by SMS. The Android backup feature does not allow to gather much information that can be interesting for a forensic analysis, but it can be used to extract SMSs and check them with MVT.
Android supports generating a backup archive of all the installed applications which supports it. However, over the years this functionality has been increasingly abandoned in favor of enabling users to remotely backup their personal data over the cloud. App developers can therefore decide to opt out from allowing the apps' data from being exported locally.
To do so, you need to connect your Android device to your computer. You will then need to [enable USB debugging](https://developer.android.com/studio/debug/dev-options#enable>) on the Android device.
At the time of writing, the Android Debug Bridge (adb) command to generate backups is still available but marked as deprecated.
If this is the first time you connect to this device, you will need to approve the authentication keys through a prompt that will appear on your Android device.
That said, most versions of Android should still allow to locally backup SMS messages, and since messages are still a prime vehicle for phishing and malware attacks, you might still want to take advantage of this functionality while it is supported.
Then you can use adb to extract the backup for SMS only with the following command:
## Generate a backup
Because `mvt-android check-backup` currently only supports checking SMS messages, you can indicate to backup only those:
```bash
adb backup com.android.providers.telephony
```
You will need to approve the backup on the phone and potentially enter a password to encrypt the backup. The backup will then be stored in a file named `backup.ab`.
In case you nonetheless wish to take a full backup, you can do so with
You will need to use [Android Backup Extractor](https://github.com/nelenkov/android-backup-extractor) to convert it to a readable file format. Make sure that java is installed on your system and use the following command:
```bash
java -jar ~/Download/abe.jar unpack backup.ab backup.tar
adb backup -all
```
## Unpack the backup
In order to reliable unpack th [Android Backup Extractor (ABE)](https://github.com/nelenkov/android-backup-extractor) to convert it to a readable file format. Make sure that java is installed on your system and use the following command:
```bash
java -jar ~/path/to/abe.jar unpack backup.ab backup.tar
tar xvf backup.tar
```
(If the backup is encrypted, the password will be asked by Android Backup Extractor).
If the backup is encrypted, ABE will prompt you to enter the password.
## Check the backup
You can then extract SMSs containing links with MVT:
```bash
$ mvt-android check-backup --output . .
$ mvt-android check-backup --output /path/to/results/ /path/to/backup/
16:18:38 INFO [mvt.android.cli] Checking ADB backup located at: .
INFO [mvt.android.modules.backup.sms] Running module SMS...
INFO [mvt.android.modules.backup.sms] Processing SMS backup
file at ./apps/com.android.providers.telephony/d_f/000
000_sms_backup
INFO [mvt.android.modules.backup.sms] Processing SMS backup file at /path/to/backup/apps/com.android.providers.telephony/d_f/000000_sms_backup
16:18:39 INFO [mvt.android.modules.backup.sms] Extracted a total of
64 SMS messages containing links
```

View File

@@ -1,15 +1,18 @@
# Downloading APKs from an Android phone
In order to use `mvt-android` you need to connect your Android device to your computer. You will then need to [enable USB debugging](https://developer.android.com/studio/debug/dev-options#enable>) on the Android device.
MVT allows to attempt to download all available installed packages (APKs) in order to further inspect them and potentially identify any which might be malicious in nature.
If this is the first time you connect to this device, you will need to approve the authentication keys through a prompt that will appear on your Android device.
Now you can launch `mvt-android` and specify the `download-apks` command and the path to the folder where you want to store the extracted data:
You can do so by launching the following command:
```bash
mvt-android download-apks --output /path/to/folder
```
It might take several minutes to complete.
!!! info
MVT will likely warn you it was unable to download certain installed packages. There is no reason to be alarmed: this is typically expected behavior when MVT attempts to download a system package it has no privileges to access.
Optionally, you can decide to enable lookups of the SHA256 hash of all the extracted APKs on [VirusTotal](https://www.virustotal.com) and/or [Koodous](https://koodous.com). While these lookups do not provide any conclusive assessment on all of the extracted APKs, they might highlight any known malicious ones:
```bash
@@ -22,3 +25,10 @@ Or, to launch all available lookups::
```bash
mvt-android download-apks --output /path/to/folder --all-checks
```
In case you have a previous extraction of APKs you want to later check against VirusTotal and Koodous, you can do so with the following arguments:
```bash
mvt-android download-apks --from-file /path/to/folder/apks.json --all-checks
```

View File

@@ -1,3 +1,20 @@
# Methodology for Android forensic
Unfortunately Android devices provide much less observability than their iOS cousins. Android stores very little diagnostic information useful to triage potential compromises, and because of this `mvt-android` capabilities are limited as well.
However, not all is lost.
## Check installed Apps
Because malware attacks over Android typically take the form of malicious or backdoored apps, the very first thing you might want to do is to extract and verify all installed Android packages and triage quickly if there are any which stand out as malicious or which might be atypical.
While it is out of the scope of this documentation to dwell into details on how to analyze Android apps, MVT does allow to easily and automatically extract information about installed apps, download copies of them, and quickly lookup services such as [VirusTotal](https://www.virustotal.com) or [Koodous](https://www.koodous.com) which might quickly indicate known bad apps.
## Check the device over Android Debug Bridge
TODO
## Check an Android Backup (SMS messages)
TODO

View File

@@ -22,7 +22,11 @@ After extracting forensics data from a device, you are also able to compare it w
mvt-ios check-iocs --iocs ~/iocs/malware.stix2 /path/to/iphone/output/
```
If you're looking for indicators of compromise for a specific piece of malware or adversary, please ask investigators or anti-malware researchers who have the relevant expertise for a STIX file.
The `--iocs` option can be invoked multiple times to let MVT import multiple STIX2 files at once. For example:
```bash
mvt-ios check-backup --iocs ~/iocs/malware1.stix --iocs ~/iocs/malware2.stix2 /path/to/backup
```
## Known repositories of STIX2 IOCs

View File

@@ -42,7 +42,8 @@ nav:
- Records extracted by mvt-ios: "ios/records.md"
- MVT for Android:
- Android Forensic Methodology: "android/methodology.md"
- Check APKs: "android/download_apks.md"
- Check an Android Backup: "android/backup.md"
- Check over ADB: "android/adb.md"
- Check an Android Backup (SMS messages): "android/backup.md"
- Download APKs: "android/download_apks.md"
- Indicators of Compromise: "iocs.md"
- License: "license.md"

View File

@@ -5,12 +5,11 @@
import logging
import os
import sys
import click
from rich.logging import RichHandler
from mvt.common.indicators import Indicators
from mvt.common.indicators import Indicators, IndicatorsFileBadFormat
from mvt.common.module import run_module, save_timeline
from .download_apks import DownloadAPKs
@@ -51,17 +50,23 @@ def cli():
help="Specify a path to a folder where you want to store the APKs")
@click.option("--from-file", "-f", type=click.Path(exists=True),
help="Instead of acquiring from phone, load an existing packages.json file for lookups (mainly for debug purposes)")
def download_apks(all_apks, virustotal, koodous, all_checks, output, from_file, serial):
@click.pass_context
def download_apks(ctx, all_apks, virustotal, koodous, all_checks, output, from_file, serial):
try:
if from_file:
download = DownloadAPKs.from_json(from_file)
else:
if output and not os.path.exists(output):
# TODO: Do we actually want to be able to run without storing any file?
if not output:
log.critical("You need to specify an output folder with --output!")
ctx.exit(1)
if not os.path.exists(output):
try:
os.makedirs(output)
except Exception as e:
log.critical("Unable to create output folder %s: %s", output, e)
sys.exit(-1)
ctx.exit(1)
download = DownloadAPKs(output_folder=output, all_apks=all_apks)
if serial:
@@ -80,7 +85,7 @@ def download_apks(all_apks, virustotal, koodous, all_checks, output, from_file,
koodous_lookup(packages)
except KeyboardInterrupt:
print("")
sys.exit(-1)
ctx.exit(1)
#==============================================================================
@@ -88,12 +93,14 @@ def download_apks(all_apks, virustotal, koodous, all_checks, output, from_file,
#==============================================================================
@cli.command("check-adb", help="Check an Android device over adb")
@click.option("--serial", "-s", type=str, help=SERIAL_HELP_MESSAGE)
@click.option("--iocs", "-i", type=click.Path(exists=True), help="Path to indicators file")
@click.option("--iocs", "-i", type=click.Path(exists=True), multiple=True,
default=[], help="Path to indicators file (can be invoked multiple times)")
@click.option("--output", "-o", type=click.Path(exists=False),
help="Specify a path to a folder where you want to store JSON results")
@click.option("--list-modules", "-l", is_flag=True, help="Print list of available modules and exit")
@click.option("--module", "-m", help="Name of a single module you would like to run instead of all")
def check_adb(iocs, output, list_modules, module, serial):
@click.pass_context
def check_adb(ctx, iocs, output, list_modules, module, serial):
if list_modules:
log.info("Following is the list of available check-adb modules:")
for adb_module in ADB_MODULES:
@@ -108,12 +115,16 @@ def check_adb(iocs, output, list_modules, module, serial):
os.makedirs(output)
except Exception as e:
log.critical("Unable to create output folder %s: %s", output, e)
sys.exit(-1)
ctx.exit(1)
if iocs:
# Pre-load indicators for performance reasons.
log.info("Loading indicators from provided file at %s", iocs)
indicators = Indicators(iocs)
indicators = Indicators(log=log)
for ioc_path in iocs:
try:
indicators.parse_stix2(ioc_path)
except IndicatorsFileBadFormat as e:
log.critical(e)
ctx.exit(1)
log.info("Loaded a total of %d indicators", indicators.ioc_count)
timeline = []
timeline_detected = []
@@ -139,15 +150,18 @@ def check_adb(iocs, output, list_modules, module, serial):
if len(timeline_detected) > 0:
save_timeline(timeline_detected, os.path.join(output, "timeline_detected.csv"))
#==============================================================================
# Check ADB backup
#==============================================================================
@cli.command("check-backup", help="Check an Android Backup")
@click.option("--serial", "-s", type=str, help=SERIAL_HELP_MESSAGE)
@click.option("--iocs", "-i", type=click.Path(exists=True), help="Path to indicators file")
@click.option("--iocs", "-i", type=click.Path(exists=True), multiple=True,
default=[], help="Path to indicators file (can be invoked multiple times)")
@click.option("--output", "-o", type=click.Path(exists=False), help=OUTPUT_HELP_MESSAGE)
@click.argument("BACKUP_PATH", type=click.Path(exists=True))
def check_backup(iocs, output, backup_path, serial):
@click.pass_context
def check_backup(ctx, iocs, output, backup_path, serial):
log.info("Checking ADB backup located at: %s", backup_path)
if output and not os.path.exists(output):
@@ -155,12 +169,16 @@ def check_backup(iocs, output, backup_path, serial):
os.makedirs(output)
except Exception as e:
log.critical("Unable to create output folder %s: %s", output, e)
sys.exit(-1)
ctx.exit(1)
if iocs:
# Pre-load indicators for performance reasons.
log.info("Loading indicators from provided file at %s", iocs)
indicators = Indicators(iocs)
indicators = Indicators(log=log)
for ioc_path in iocs:
try:
indicators.parse_stix2(ioc_path)
except IndicatorsFileBadFormat as e:
log.critical(e)
ctx.exit(1)
log.info("Loaded a total of %d indicators", indicators.ioc_count)
if os.path.isfile(backup_path):
log.critical("The path you specified is a not a folder!")
@@ -168,7 +186,7 @@ def check_backup(iocs, output, backup_path, serial):
if os.path.basename(backup_path) == "backup.ab":
log.info("You can use ABE (https://github.com/nelenkov/android-backup-extractor) " \
"to extract 'backup.ab' files!")
sys.exit(-1)
ctx.exit(1)
for module in BACKUP_MODULES:
m = module(base_folder=backup_path, output_folder=output,

View File

@@ -10,6 +10,7 @@ import os
import pkg_resources
from tqdm import tqdm
from mvt.common.module import InsufficientPrivileges
from mvt.common.utils import get_sha256_from_file_path
from .modules.adb.base import AndroidExtraction
@@ -58,8 +59,8 @@ class DownloadAPKs(AndroidExtraction):
@classmethod
def from_json(cls, json_path):
"""Initialize this class from an existing packages.json file.
:param json_path: Path to the packages.json file to parse.
"""Initialize this class from an existing apks.json file.
:param json_path: Path to the apks.json file to parse.
"""
with open(json_path, "r") as handle:
data = json.load(handle)
@@ -139,6 +140,11 @@ class DownloadAPKs(AndroidExtraction):
miniters=1) as pp:
self._adb_download(remote_path, local_path,
progress_callback=pp.update_to)
except InsufficientPrivileges:
log.warn("Unable to pull package file from %s: insufficient privileges, it might be a system app",
remote_path)
self._adb_reconnect()
return None
except Exception as e:
log.exception("Failed to pull package file from %s: %s",
remote_path, e)
@@ -196,7 +202,7 @@ class DownloadAPKs(AndroidExtraction):
def save_json(self):
"""Save the results to the package.json file.
"""
json_path = os.path.join(self.output_folder, "packages.json")
json_path = os.path.join(self.output_folder, "apks.json")
packages = []
for package in self.packages:
packages.append(package.__dict__)

View File

@@ -14,7 +14,8 @@ import time
from adb_shell.adb_device import AdbDeviceTcp, AdbDeviceUsb
from adb_shell.auth.keygen import keygen, write_public_keyfile
from adb_shell.auth.sign_pythonrsa import PythonRSASigner
from adb_shell.exceptions import AdbCommandFailureException, DeviceAuthError
from adb_shell.exceptions import (AdbCommandFailureException, DeviceAuthError,
UsbReadFailedError)
from usb1 import USBErrorAccess, USBErrorBusy
from mvt.common.module import InsufficientPrivileges, MVTModule
@@ -77,9 +78,14 @@ class AndroidExtraction(MVTModule):
except DeviceAuthError:
log.error("You need to authorize this computer on the Android device. Retrying in 5 seconds...")
time.sleep(5)
except Exception as e:
log.critical(e)
except UsbReadFailedError:
log.error("Unable to connect to the device over USB. Try to unplug, plug the device and start again.")
sys.exit(-1)
except OSError as e:
if e.errno == 113 and self.serial:
log.critical("Unable to connect to the device %s: did you specify the correct IP addres?",
self.serial)
sys.exit(-1)
else:
break

View File

@@ -66,9 +66,15 @@ class Packages(AndroidExtraction):
fields = line.split()
file_name, package_name = fields[0].split(":")[1].rsplit("=", 1)
installer = fields[1].split("=")[1].strip()
if installer == "null":
try:
installer = fields[1].split("=")[1].strip()
except IndexError:
installer = None
else:
if installer == "null":
installer = None
uid = fields[2].split(":")[1].strip()
dumpsys = self._adb_command(f"dumpsys package {package_name} | grep -A2 timeStamp").split("\n")
@@ -106,6 +112,13 @@ class Packages(AndroidExtraction):
if result["package_name"] == package_name:
self.results[i][cmd["field"]] = True
for result in self.results:
if result["system"]:
continue
self.log.info("Found non-system package with name \"%s\" installed by \"%s\" on %s",
result["package_name"], result["installer"], result["timestamp"])
self.log.info("Extracted at total of %d installed package names",
len(self.results))

View File

@@ -17,50 +17,52 @@ class Indicators:
functions to compare extracted artifacts to the indicators.
"""
def __init__(self, file_path, log=None):
self.file_path = file_path
with open(self.file_path, "r") as handle:
try:
self.data = json.load(handle)
except json.decoder.JSONDecodeError:
raise IndicatorsFileBadFormat("Unable to parse STIX2 indicators file, the file seems malformed or in the wrong format")
def __init__(self, log=None):
self.log = log
self.ioc_domains = []
self.ioc_processes = []
self.ioc_emails = []
self.ioc_files = []
self._parse_stix_file()
self.ioc_count = 0
def _parse_stix_file(self):
"""Extract IOCs of given type from STIX2 definitions.
def _add_indicator(self, ioc, iocs_list):
if ioc not in iocs_list:
iocs_list.append(ioc)
self.ioc_count += 1
def parse_stix2(self, file_path):
"""Extract indicators from a STIX2 file.
"""
for entry in self.data["objects"]:
self.log.info("Parsing STIX2 indicators file at path %s",
file_path)
with open(file_path, "r") as handle:
try:
if entry["type"] != "indicator":
continue
except KeyError:
data = json.load(handle)
except json.decoder.JSONDecodeError:
raise IndicatorsFileBadFormat("Unable to parse STIX2 indicators file, the file seems malformed or in the wrong format")
for entry in data.get("objects", []):
if entry.get("type", "") != "indicator":
continue
key, value = entry["pattern"].strip("[]").split("=")
key, value = entry.get("pattern", "").strip("[]").split("=")
value = value.strip("'")
if key == "domain-name:value":
# We force domain names to lower case.
value = value.lower()
if value not in self.ioc_domains:
self.ioc_domains.append(value)
self._add_indicator(ioc=value.lower(),
iocs_list=self.ioc_domains)
elif key == "process:name":
if value not in self.ioc_processes:
self.ioc_processes.append(value)
self._add_indicator(ioc=value,
iocs_list=self.ioc_processes)
elif key == "email-addr:value":
# We force email addresses to lower case.
value = value.lower()
if value not in self.ioc_emails:
self.ioc_emails.append(value)
self._add_indicator(ioc=value.lower(),
iocs_list=self.ioc_emails)
elif key == "file:name":
if value not in self.ioc_files:
self.ioc_files.append(value)
self._add_indicator(ioc=value,
iocs_list=self.ioc_files)
def check_domain(self, url):
# TODO: If the IOC domain contains a subdomain, it is not currently

View File

@@ -121,7 +121,8 @@ def extract_key(password, backup_path, key_file):
# Command: check-backup
#==============================================================================
@cli.command("check-backup", help="Extract artifacts from an iTunes backup")
@click.option("--iocs", "-i", type=click.Path(exists=True), help="Path to indicators file")
@click.option("--iocs", "-i", type=click.Path(exists=True), multiple=True,
default=[], help="Path to indicators file (can be invoked multiple time)")
@click.option("--output", "-o", type=click.Path(exists=False), help=OUTPUT_HELP_MESSAGE)
@click.option("--fast", "-f", is_flag=True, help="Avoid running time/resource consuming features")
@click.option("--list-modules", "-l", is_flag=True, help="Print list of available modules and exit")
@@ -145,14 +146,14 @@ def check_backup(ctx, iocs, output, fast, backup_path, list_modules, module):
log.critical("Unable to create output folder %s: %s", output, e)
ctx.exit(1)
if iocs:
# Pre-load indicators for performance reasons.
log.info("Loading indicators from provided file at: %s", iocs)
indicators = Indicators(log=log)
for ioc_path in iocs:
try:
indicators = Indicators(iocs)
indicators.parse_stix2(ioc_path)
except IndicatorsFileBadFormat as e:
log.critical(e)
ctx.exit(1)
log.info("Loaded a total of %d indicators", indicators.ioc_count)
timeline = []
timeline_detected = []
@@ -165,8 +166,8 @@ def check_backup(ctx, iocs, output, fast, backup_path, list_modules, module):
m.is_backup = True
if iocs:
indicators.log = m.log
m.indicators = indicators
m.indicators.log = m.log
run_module(m)
timeline.extend(m.timeline)
@@ -183,7 +184,8 @@ def check_backup(ctx, iocs, output, fast, backup_path, list_modules, module):
# Command: check-fs
#==============================================================================
@cli.command("check-fs", help="Extract artifacts from a full filesystem dump")
@click.option("--iocs", "-i", type=click.Path(exists=True), help="Path to indicators file")
@click.option("--iocs", "-i", type=click.Path(exists=True), multiple=True,
default=[], help="Path to indicators file (can be invoked multiple time)")
@click.option("--output", "-o", type=click.Path(exists=False), help=OUTPUT_HELP_MESSAGE)
@click.option("--fast", "-f", is_flag=True, help="Avoid running time/resource consuming features")
@click.option("--list-modules", "-l", is_flag=True, help="Print list of available modules and exit")
@@ -207,14 +209,14 @@ def check_fs(ctx, iocs, output, fast, dump_path, list_modules, module):
log.critical("Unable to create output folder %s: %s", output, e)
ctx.exit(1)
if iocs:
# Pre-load indicators for performance reasons.
log.info("Loading indicators from provided file at: %s", iocs)
indicators = Indicators(log=log)
for ioc_path in iocs:
try:
indicators = Indicators(iocs)
indicators.parse_stix2(ioc_path)
except IndicatorsFileBadFormat as e:
log.critical(e)
ctx.exit(1)
log.info("Loaded a total of %d indicators", indicators.ioc_count)
timeline = []
timeline_detected = []
@@ -228,8 +230,8 @@ def check_fs(ctx, iocs, output, fast, dump_path, list_modules, module):
m.is_fs_dump = True
if iocs:
indicators.log = m.log
m.indicators = indicators
m.indicators.log = m.log
run_module(m)
timeline.extend(m.timeline)
@@ -246,8 +248,8 @@ def check_fs(ctx, iocs, output, fast, dump_path, list_modules, module):
# Command: check-iocs
#==============================================================================
@cli.command("check-iocs", help="Compare stored JSON results to provided indicators")
@click.option("--iocs", "-i", required=True, type=click.Path(exists=True),
help="Path to indicators file")
@click.option("--iocs", "-i", type=click.Path(exists=True), multiple=True,
default=[], required=True, help="Path to indicators file (can be invoked multiple time)")
@click.option("--list-modules", "-l", is_flag=True, help="Print list of available modules and exit")
@click.option("--module", "-m", help="Name of a single module you would like to run instead of all")
@click.argument("FOLDER", type=click.Path(exists=True))
@@ -267,14 +269,14 @@ def check_iocs(ctx, iocs, list_modules, module, folder):
log.info("Checking stored results against provided indicators...")
# Pre-load indicators for performance reasons.
log.info("Loading indicators from provided file at: %s", iocs)
try:
indicators = Indicators(iocs)
except IndicatorsFileBadFormat as e:
log.critical(e)
ctx.exit(1)
indicators = Indicators(log=log)
for ioc_path in iocs:
try:
indicators.parse_stix2(ioc_path)
except IndicatorsFileBadFormat as e:
log.critical(e)
ctx.exit(1)
log.info("Loaded a total of %d indicators", indicators.ioc_count)
for file_name in os.listdir(folder):
name_only, ext = os.path.splitext(file_name)
@@ -293,8 +295,8 @@ def check_iocs(ctx, iocs, list_modules, module, folder):
m = iocs_module.from_json(file_path,
log=logging.getLogger(iocs_module.__module__))
indicators.log = m.log
m.indicators = indicators
m.indicators.log = m.log
try:
m.check_indicators()

View File

@@ -8,7 +8,7 @@ import os
from setuptools import find_packages, setup
__package_name__ = "mvt"
__version__ = "1.2.0"
__version__ = "1.2.1"
__description__ = "Mobile Verification Toolkit"
this_directory = os.path.abspath(os.path.dirname(__file__))