1
mirror of https://github.com/mvt-project/mvt synced 2025-10-21 22:42:15 +02:00

Compare commits

...

65 Commits

Author SHA1 Message Date
Tek
7a1ed628ef Create dependabot.yml 2025-06-20 11:16:08 +02:00
dependabot[bot]
c9ac12f336 Bump requests from 2.32.2 to 2.32.4 (#642)
Bumps [requests](https://github.com/psf/requests) from 2.32.2 to 2.32.4.
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.32.2...v2.32.4)

---
updated-dependencies:
- dependency-name: requests
  dependency-version: 2.32.4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-19 23:55:42 +02:00
Tek
486e3e7e9b Update global_preferences.py (#641)
Added a warning for lockdown mode when the property is not present
---------

Co-authored-by: makitos666 <makitos666@gmail.com>
2025-06-19 23:51:59 +02:00
besendorf
be1fc3bd8b Update NSKeyedUnarchiver (#636) 2025-06-12 22:42:02 +02:00
Tek
4757cff262 Fixes date parsing issue in tombstones (#635) 2025-06-12 20:49:31 +02:00
Tek
61f51caf31 Freeze versions and bump version (#632)
* Freeze versions and bump version
* Drops support for python below 3.10
2025-06-12 16:33:15 +02:00
besendorf
511063fd0e Update pyproject.toml (#630) 2025-06-04 13:00:04 +02:00
scribblemaniac
88bc5672cb Upgrade main dockerfile runtime to ubuntu:24.04 (#619)
Co-authored-by: Tek <tek@randhome.io>
2025-05-14 11:34:40 +02:00
github-actions[bot]
0fce0acf7a Add new iOS versions and build numbers (#626)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-05-14 11:12:13 +02:00
github-actions[bot]
61f95d07d3 Add new iOS versions and build numbers (#625)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-05-12 22:37:46 +02:00
ping2A
3dedd169c4 Fix issue #574 for a module without IOCs output (#620)
* Fix issue #574 for a module without IOCs output
2025-04-30 10:30:39 +02:00
Tek
e34e03d3a3 Fixes Android Dumpsys ADB parsing issue 2025-04-18 17:43:08 +02:00
github-actions[bot]
34374699ce Add new iOS versions and build numbers (#622)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-04-17 09:46:17 +02:00
github-actions[bot]
cf5aa7c89f Add new iOS versions and build numbers (#618)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-04-01 16:04:06 +02:00
Donncha Ó Cearbhaill
2766739512 Fix bug where default values were dropped when parsing protobuf tombstones (#617) 2025-03-11 14:10:34 +01:00
cacu
9c84afb4b0 Update logo.py (#615)
add instructions to update mvt via pipx
2025-03-11 13:46:59 +01:00
Donncha Ó Cearbhaill
80fc8bd879 Fix YAML format (#611) 2025-02-21 15:48:00 +01:00
Donncha Ó Cearbhaill
ca41f7f106 Always open automatic PRs as drafts (#609) 2025-02-21 15:35:06 +01:00
github-actions[bot]
55ddd86ad5 Add new iOS versions and build numbers (#607)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-02-21 15:24:27 +01:00
Donncha Ó Cearbhaill
b184eeedf4 Handle XML encoded ADB keystore and fix parsing bugs (#605) 2025-02-07 02:00:24 +01:00
Donncha Ó Cearbhaill
4e97e85350 Load Android device timezone info and add additional file modification logs (#567)
* Use local timestamp for Files module timeline.

Most other Android timestamps appear to be local time. The
results timeline is more useful if all the timestamps
are consistent. I would prefer to use UTC, but that would
mean converting all the other timestamps to UTC as well. We probably
do not have sufficient information to do that accurately,
especially if the device is moving between timezones..

* Add file timestamp modules to add logs into timeline

* Handle case were we cannot load device timezone

* Fix crash if prop file does not exist

* Move _get_file_modification_time to BugReportModule

* Add backport for timezone and fix Tombstone module to use local time.

* Fix import for backported Zoneinfo

* Fix ruff error
2025-02-06 20:51:15 +01:00
Donncha Ó Cearbhaill
e5865b166e Merge pull request #568 from mvt-project/feature/tombstone-parser
Add parser for Android tombstone files
2025-02-06 20:15:21 +01:00
Donncha Ó Cearbhaill
a2dabb4267 Fix generate-proto-parsers Makefile command 2025-02-06 20:11:54 +01:00
Donncha Ó Cearbhaill
b7595b62eb Add initial tombstone parser
This supports parsing tombstone files from Android bugreports. The parser
can load both the legacy text format and the new binary protobuf format.
2025-02-06 20:07:05 +01:00
Donncha Ó Cearbhaill
02c02ca15c Merge branch 'main' into feature/tombstone-parser 2025-02-03 18:44:00 +01:00
Donncha Ó Cearbhaill
6da33394fe Merge pull request #592 from mvt-project/feature/config-file
Reworking handling of config options
2025-01-30 13:32:53 +01:00
Donncha Ó Cearbhaill
086871e21d Merge branch 'main' into feature/config-file 2025-01-30 13:15:28 +01:00
Donncha Ó Cearbhaill
f32830c649 Merge pull request #603 from mvt-project/feature/add-suspicious-android-setting
Add additional Android security warnings
2025-01-30 13:12:14 +01:00
Donncha Ó Cearbhaill
edcad488ab Merge branch 'main' into feature/add-suspicious-android-setting 2025-01-30 13:10:00 +01:00
Donncha Ó Cearbhaill
43901c96a0 Add improved heuristic detections to AppOps module 2025-01-30 13:02:26 +01:00
Donncha Ó Cearbhaill
0962383b46 Alert on potentially suspicious permissions from ADB 2025-01-30 11:48:19 +01:00
Donncha Ó Cearbhaill
34cd08fd9a Add additional Android security setting to warn on 2025-01-30 11:35:18 +01:00
github-actions[bot]
579b53f7ec Add new iOS versions and build numbers (#602)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-01-28 01:27:17 +01:00
Rory Flynn
dbb80d6320 Mark release 2.6.0 (#601) 2025-01-27 15:41:41 +01:00
Donncha Ó Cearbhaill
0fbf24e82a Merge branch 'main' into feature/config-file 2025-01-14 14:33:40 +01:00
Rory Flynn
a2493baead Documentation tweaks (#599)
* Adds link in install instructions to the command completion docs added in #597
* Small visual tweaks
2025-01-14 13:12:10 +01:00
Nim
0dc6228a59 Add command completion docs (#410) (#597)
Co-authored-by: Rory Flynn <75283103+roaree@users.noreply.github.com>
2025-01-14 12:04:07 +01:00
Rory Flynn
6e230bdb6a Autofix for ruff (#598) 2025-01-14 12:02:10 +01:00
Tek
2aa76c8a1c Fixes a bug on recent phones not having WIFI column in net usage (#580)
Co-authored-by: Donncha Ó Cearbhaill <donncha.ocearbhaill@amnesty.org>
Co-authored-by: Rory Flynn <75283103+roaree@users.noreply.github.com>
2025-01-07 12:48:35 +01:00
github-actions[bot]
7d6dc9e6dc Add new iOS versions and build numbers (#595)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2025-01-07 12:07:57 +01:00
Donncha Ó Cearbhaill
458195a0ab Fix optional typing syntax for Python 3.8 2024-12-25 00:28:02 +00:00
Donncha Ó Cearbhaill
52e854b8b7 Add missing import 2024-12-25 00:23:36 +00:00
Donncha Ó Cearbhaill
0f1eec3971 Add Pydantic dependencies 2024-12-25 00:21:42 +00:00
Donncha Ó Cearbhaill
f4425865c0 Add missed modules using updated settings module 2024-12-25 00:14:14 +00:00
Donncha Ó Cearbhaill
28c0c86c4e Update MVT code to use config file rather than raw env variables 2024-12-25 00:09:29 +00:00
Donncha Ó Cearbhaill
154e6dab15 Add config file parser for MVT 2024-12-24 23:30:18 +00:00
Donncha Ó Cearbhaill
0c73e3e8fa Merge pull request #587 from mvt-project/feature/uninstalled-apps
Add a module to parse uninstalled apps from dumpsys data
2024-12-16 00:03:23 +01:00
Donncha Ó Cearbhaill
9b5f2d89d5 Merge branch 'main' into feature/uninstalled-apps 2024-12-16 00:00:12 +01:00
Donncha Ó Cearbhaill
3da61c8da8 Fix ruff checks 2024-12-15 23:22:36 +01:00
Tek
5b2fe3baec Reorganize code in iOS app module (#586) 2024-12-14 10:04:47 +01:00
Donncha Ó Cearbhaill
a3a7789547 Merge pull request #584 from mvt-project/enhance-community-guidelines
Update MVT contributor guidelines
2024-12-13 23:01:58 +01:00
Donncha Ó Cearbhaill
d3fcc686ff Update contribution guidelines 2024-12-13 22:45:41 +01:00
github-actions[bot]
4bcc0e5f27 Add new iOS versions and build numbers (#583)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2024-12-12 14:43:59 +01:00
tes
9d81b5bfa8 Add a module to parse uninstalled apps from dumpsys data, for both bugreport and AndroidQF output, and match them against package name IoCs. 2024-12-11 16:47:19 -03:00
github-actions[bot]
22fce280af Add new iOS versions and build numbers (#572)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2024-11-20 11:02:09 +01:00
Donncha Ó Cearbhaill
4739d8853e Merge pull request #570 from mvt-project/fix/files-detection-bug
Fix error to due extra equal character in Files detection
2024-10-31 20:04:33 +01:00
Donncha Ó Cearbhaill
ace01ff7fb Merge branch 'main' into fix/files-detection-bug 2024-10-31 19:59:53 +01:00
Donncha Ó Cearbhaill
7e4f0aec4d Fix error to due extra equal character in Files detection 2024-10-31 19:59:29 +01:00
github-actions[bot]
57647583cc Add new iOS versions and build numbers (#569)
Co-authored-by: DonnchaC <DonnchaC@users.noreply.github.com>
2024-10-29 04:17:03 +01:00
Donncha Ó Cearbhaill
8e895d3d07 Remove protobuf compiler dependency, only needed for dev 2024-10-28 13:10:37 +01:00
Donncha Ó Cearbhaill
bc09e2a394 Initial tests for tombstone parsing 2024-10-28 10:51:58 +01:00
Donncha Ó Cearbhaill
2d0de088dd Add generated protobuf parser 2024-10-28 10:38:19 +01:00
Donncha Ó Cearbhaill
8694e7a047 Add protobuf parser generation 2024-10-28 10:37:30 +01:00
Donncha Ó Cearbhaill
9b41ba99aa WIP: initial tombstone modules 2024-10-28 10:34:53 +01:00
Donncha Ó Cearbhaill
cd99b293ed Merge pull request #563 from mvt-project/feature/add-package-detections
Add additional detections for suspicious packages
2024-10-24 17:37:30 +02:00
74 changed files with 2977 additions and 148 deletions

11
.github/dependabot.yml vendored Normal file
View File

@@ -0,0 +1,11 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
version: 2
updates:
- package-ecosystem: "pip" # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "weekly"

View File

@@ -12,7 +12,7 @@ jobs:
strategy:
fail-fast: false
matrix:
python-version: ['3.8', '3.9', '3.10'] # , '3.11']
python-version: ['3.10', '3.11', '3.12', '3.13']
steps:
- uses: actions/checkout@v4
@@ -35,4 +35,4 @@ jobs:
if: github.event_name == 'pull_request'
with:
pytest-coverage-path: ./pytest-coverage.txt
junitxml-path: ./pytest.xml
junitxml-path: ./pytest.xml

View File

@@ -21,6 +21,7 @@ jobs:
title: '[auto] Update iOS releases and versions'
commit-message: Add new iOS versions and build numbers
branch: auto/add-new-ios-releases
draft: true
body: |
This is an automated pull request to update the iOS releases and version numbers.
add-paths: |

View File

@@ -1,19 +1,65 @@
# Contributing
# Contributing to Mobile Verification Toolkit (MVT)
Thank you for your interest in contributing to Mobile Verification Toolkit (MVT)! Your help is very much appreciated.
We greatly appreciate contributions to MVT!
Your involvement, whether through identifying issues, improving functionality, or enhancing documentation, is very much appreciated. To ensure smooth collaboration and a welcoming environment, we've outlined some key guidelines for contributing below.
## Getting started
Contributing to an open-source project like MVT might seem overwhelming at first, but we're here to support you!
Whether you're a technologist, a frontline human rights defender, a field researcher, or someone new to consensual spyware forensics, there are many ways to make meaningful contributions.
Here's how you can get started:
1. **Explore the codebase:**
- Browse the repository to get familar with MVT. Many MVT modules are simple in functionality and easy to understand.
- Look for `TODO:` or `FIXME:` comments in the code for areas that need attention.
2. **Check Github issues:**
- Look for issues tagged with ["help wanted"](https://github.com/mvt-project/mvt/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) or ["good first issue"](https://github.com/mvt-project/mvt/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) to find tasks that are beginner-friendly or where input from the community would be helpful.
3. **Ask for guidance:**
- If you're unsure where to start, feel free to open a [discussion](https://github.com/mvt-project/mvt/discussions) or comment on an issue.
## How to contribute:
1. **Report issues:**
- Found a bug? Please check existing issues to see if it's already reported. If not, open a new issue. Mobile operating systems and databases are constantly evolving, an new errors may appear spontaniously in new app versions.
**Please provide as much information as possible about the prodblem including: any error messages, steps to reproduce the problem, and any logs or screenshots that can help.**
## Where to start
2. **Suggest features:**
- If you have an idea for new functionality, create a feature request issue and describe your proposal.
Starting to contribute to a somewhat complex project like MVT might seem intimidating. Unless you have specific ideas of new functionality you would like to submit, some good starting points are searching for `TODO:` and `FIXME:` comments throughout the code. Alternatively you can check if any GitHub issues existed marked with the ["help wanted"](https://github.com/mvt-project/mvt/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) tag.
3. **Submit code:**
- Fork the repository and create a new branch for your changes.
- Ensure your changes align with the code style guidelines (see below).
- Open a pull request (PR) with a clear description of your changes and link it to any relevant issues.
4. **Documentation contributions:**
- Improving documentation is just as valuable as contributing code! If you notice gaps or inaccuracies in the documentation, feel free to submit changes or suggest updates.
## Code style
Please follow these code style guidelines for consistency and readability:
When contributing code to
- **Indentation**: use 4 spaces per tab.
- **Quotes**: Use double quotes (`"`) by default. Use single quotes (`'`) for nested strings instead of escaping (`\"`), or when using f-formatting.
- **Maximum line length**:
- Aim for lines no longer than 80 characters.
- Exceptions are allowed for long log lines or strings, which may extend up to 100 characters.
- Wrap lines that exceed 100 characters.
- **Indentation**: we use 4-spaces tabs.
Follow [PEP 8 guidelines](https://peps.python.org/pep-0008/) for indentation and overall Python code style. All MVT code is automatically linted with [Ruff](https://github.com/astral-sh/ruff) before merging.
- **Quotes**: we use double quotes (`"`) as a default. Single quotes (`'`) can be favored with nested strings instead of escaping (`\"`), or when using f-formatting.
Please check your code before opening a pull request by running `make ruff`
- **Maximum line length**: we strongly encourage to respect a 80 characters long lines and to follow [PEP8 indentation guidelines](https://peps.python.org/pep-0008/#indentation) when having to wrap. However, if breaking at 80 is not possible or is detrimental to the readability of the code, exceptions are tolerated. For example, long log lines, or long strings can be extended to 100 characters long. Please hard wrap anything beyond 100 characters.
## Community and support
We aim to create a supportive and collaborative environment for all contributors. If you run into any challenges, feel free to reach out through the discussions or issues section of the repository.
Your contributions, big or small, help improve MVT and are always appreciated.

View File

@@ -103,7 +103,7 @@ RUN git clone https://github.com/libimobiledevice/usbmuxd && cd usbmuxd \
# Create main image
FROM ubuntu:22.04 as main
FROM ubuntu:24.04 as main
LABEL org.opencontainers.image.url="https://mvt.re"
LABEL org.opencontainers.image.documentation="https://docs.mvt.re"
@@ -135,8 +135,7 @@ COPY --from=build-usbmuxd /build /
COPY . mvt/
RUN apt-get update \
&& apt-get install -y git python3-pip \
&& PIP_NO_CACHE_DIR=1 pip3 install --upgrade pip \
&& PIP_NO_CACHE_DIR=1 pip3 install ./mvt \
&& PIP_NO_CACHE_DIR=1 pip3 install --break-system-packages ./mvt \
&& apt-get remove -y python3-pip git && apt-get autoremove -y \
&& rm -rf /var/lib/apt/lists/* \
&& rm -rf mvt

View File

@@ -25,6 +25,11 @@ install:
test-requirements:
python3 -m pip install --upgrade -r test-requirements.txt
generate-proto-parsers:
# Generate python parsers for protobuf files
PROTO_FILES=$$(find src/mvt/android/parsers/proto/ -iname "*.proto"); \
protoc -Isrc/mvt/android/parsers/proto/ --python_betterproto_out=src/mvt/android/parsers/proto/ $$PROTO_FILES
clean:
rm -rf $(PWD)/build $(PWD)/dist $(PWD)/src/mvt.egg-info

View File

@@ -0,0 +1,43 @@
# Command Completion
MVT utilizes the [Click](https://click.palletsprojects.com/en/stable/) library for creating its command line interface.
Click provides tab completion support for Bash (version 4.4 and up), Zsh, and Fish.
To enable it, you need to manually register a special function with your shell, which varies depending on the shell you are using.
The following describes how to generate the command completion scripts and add them to your shell configuration.
> **Note: You will need to start a new shell for the changes to take effect.**
### For Bash
```bash
# Generates bash completion scripts
echo "$(_MVT_IOS_COMPLETE=bash_source mvt-ios)" > ~/.mvt-ios-complete.bash &&
echo "$(_MVT_ANDROID_COMPLETE=bash_source mvt-android)" > ~/.mvt-android-complete.bash
```
Add the following to `~/.bashrc`:
```bash
# source mvt completion scripts
. ~/.mvt-ios-complete.bash && . ~/.mvt-android-complete.bash
```
### For Zsh
```bash
# Generates zsh completion scripts
echo "$(_MVT_IOS_COMPLETE=zsh_source mvt-ios)" > ~/.mvt-ios-complete.zsh &&
echo "$(_MVT_ANDROID_COMPLETE=zsh_source mvt-android)" > ~/.mvt-android-complete.zsh
```
Add the following to `~/.zshrc`:
```bash
# source mvt completion scripts
. ~/.mvt-ios-complete.zsh && . ~/.mvt-android-complete.zsh
```
For more information, visit the official [Click Docs](https://click.palletsprojects.com/en/stable/shell-completion/#enabling-completion).

View File

@@ -98,3 +98,7 @@ You now should have the `mvt-ios` and `mvt-android` utilities installed.
**Notes:**
1. The `--force` flag is necessary to force the reinstallation of the package.
2. To revert to using a PyPI version, it will be necessary to `pipx uninstall mvt` first.
## Setting up command completions
See ["Command completions"](command_completion.md)

View File

@@ -19,21 +19,26 @@ classifiers = [
"Programming Language :: Python"
]
dependencies = [
"click >=8.1.3",
"rich >=12.6.0",
"tld >=0.12.6",
"requests >=2.28.1",
"simplejson >=3.17.6",
"packaging >=21.3",
"appdirs >=1.4.4",
"iOSbackup >=0.9.923",
"adb-shell[usb] >=0.4.3",
"libusb1 >=3.0.0",
"cryptography >=42.0.5",
"pyyaml >=6.0",
"pyahocorasick >= 2.0.0",
"click==8.2.1",
"rich==14.0.0",
"tld==0.13.1",
"requests==2.32.4",
"simplejson==3.20.1",
"packaging==25.0",
"appdirs==1.4.4",
"iOSbackup==0.9.925",
"adb-shell[usb]==0.4.4",
"libusb1==3.3.1",
"cryptography==45.0.3",
"PyYAML>=6.0.2",
"pyahocorasick==2.1.0",
"betterproto==1.2.5",
"pydantic==2.11.5",
"pydantic-settings==2.9.1",
"NSKeyedUnArchiver==1.5.2",
"python-dateutil==2.9.0.post0",
]
requires-python = ">= 3.8"
requires-python = ">= 3.10"
[project.urls]
homepage = "https://docs.mvt.re/en/latest/"
@@ -99,4 +104,4 @@ where = ["src"]
mvt = ["ios/data/*.json"]
[tool.setuptools.dynamic]
version = {attr = "mvt.common.version.MVT_VERSION"}
version = {attr = "mvt.common.version.MVT_VERSION"}

View File

@@ -4,13 +4,14 @@
# https://license.mvt.re/1.1/
import base64
import binascii
import hashlib
from .artifact import AndroidArtifact
class DumpsysADBArtifact(AndroidArtifact):
multiline_fields = ["user_keys"]
multiline_fields = ["user_keys", "keystore"]
def indented_dump_parser(self, dump_data):
"""
@@ -67,14 +68,38 @@ class DumpsysADBArtifact(AndroidArtifact):
return res
def parse_xml(self, xml_data):
"""
Parse XML data from dumpsys ADB output
"""
import xml.etree.ElementTree as ET
keystore = []
keystore_root = ET.fromstring(xml_data)
for adb_key in keystore_root.findall("adbKey"):
key_info = self.calculate_key_info(adb_key.get("key").encode("utf-8"))
key_info["last_connected"] = adb_key.get("lastConnection")
keystore.append(key_info)
return keystore
@staticmethod
def calculate_key_info(user_key: bytes) -> str:
key_base64, user = user_key.split(b" ", 1)
key_raw = base64.b64decode(key_base64)
key_fingerprint = hashlib.md5(key_raw).hexdigest().upper()
key_fingerprint_colon = ":".join(
[key_fingerprint[i : i + 2] for i in range(0, len(key_fingerprint), 2)]
)
if b" " in user_key:
key_base64, user = user_key.split(b" ", 1)
else:
key_base64, user = user_key, b""
try:
key_raw = base64.b64decode(key_base64)
key_fingerprint = hashlib.md5(key_raw).hexdigest().upper()
key_fingerprint_colon = ":".join(
[key_fingerprint[i : i + 2] for i in range(0, len(key_fingerprint), 2)]
)
except binascii.Error:
# Impossible to parse base64
key_fingerprint_colon = ""
return {
"user": user.decode("utf-8"),
"fingerprint": key_fingerprint_colon,
@@ -115,8 +140,24 @@ class DumpsysADBArtifact(AndroidArtifact):
if parsed.get("debugging_manager") is None:
self.log.error("Unable to find expected ADB entries in dumpsys output") # noqa
return
# Keystore can be in different levels, as the basic parser
# is not always consistent due to different dumpsys formats.
if parsed.get("keystore"):
keystore_data = b"\n".join(parsed["keystore"])
elif parsed["debugging_manager"].get("keystore"):
keystore_data = b"\n".join(parsed["debugging_manager"]["keystore"])
else:
parsed = parsed["debugging_manager"]
keystore_data = None
# Keystore is in XML format on some devices and we need to parse it
if keystore_data and keystore_data.startswith(b"<?xml"):
parsed["debugging_manager"]["keystore"] = self.parse_xml(keystore_data)
else:
# Keystore is not XML format
parsed["debugging_manager"]["keystore"] = keystore_data
parsed = parsed["debugging_manager"]
# Calculate key fingerprints for better readability
key_info = []

View File

@@ -11,6 +11,10 @@ from mvt.common.utils import convert_datetime_to_iso
from .artifact import AndroidArtifact
RISKY_PERMISSIONS = ["REQUEST_INSTALL_PACKAGES"]
RISKY_PACKAGES = ["com.android.shell"]
class DumpsysAppopsArtifact(AndroidArtifact):
"""
Parser for dumpsys app ops info
@@ -45,15 +49,39 @@ class DumpsysAppopsArtifact(AndroidArtifact):
self.detected.append(result)
continue
detected_permissions = []
for perm in result["permissions"]:
if (
perm["name"] == "REQUEST_INSTALL_PACKAGES"
and perm["access"] == "allow"
perm["name"] in RISKY_PERMISSIONS
# and perm["access"] == "allow"
):
self.log.info(
"Package %s with REQUEST_INSTALL_PACKAGES " "permission",
result["package_name"],
)
detected_permissions.append(perm)
for entry in sorted(perm["entries"], key=lambda x: x["timestamp"]):
self.log.warning(
"Package '%s' had risky permission '%s' set to '%s' at %s",
result["package_name"],
perm["name"],
entry["access"],
entry["timestamp"],
)
elif result["package_name"] in RISKY_PACKAGES:
detected_permissions.append(perm)
for entry in sorted(perm["entries"], key=lambda x: x["timestamp"]):
self.log.warning(
"Risky package '%s' had '%s' permission set to '%s' at %s",
result["package_name"],
perm["name"],
entry["access"],
entry["timestamp"],
)
if detected_permissions:
# We clean the result to only include the risky permission, otherwise the timeline
# will be polluted with all the other irrelevant permissions
cleaned_result = result.copy()
cleaned_result["permissions"] = detected_permissions
self.detected.append(cleaned_result)
def parse(self, output: str) -> None:
self.results: List[Dict[str, Any]] = []
@@ -121,11 +149,16 @@ class DumpsysAppopsArtifact(AndroidArtifact):
if line.startswith(" "):
# Permission entry like:
# Reject: [fg-s]2021-05-19 22:02:52.054 (-314d1h25m2s33ms)
access_type = line.split(":")[0].strip()
if access_type not in ["Access", "Reject"]:
# Skipping invalid access type. Some entries are not in the format we expect
continue
if entry:
perm["entries"].append(entry)
entry = {}
entry["access"] = line.split(":")[0].strip()
entry["access"] = access_type
entry["type"] = line[line.find("[") + 1 : line.find("]")]
try:

View File

@@ -16,8 +16,7 @@ class DumpsysPackagesArtifact(AndroidArtifact):
for result in self.results:
if result["package_name"] in ROOT_PACKAGES:
self.log.warning(
"Found an installed package related to "
'rooting/jailbreaking: "%s"',
'Found an installed package related to rooting/jailbreaking: "%s"',
result["package_name"],
)
self.detected.append(result)

View File

@@ -0,0 +1,42 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
from .artifact import AndroidArtifact
class DumpsysPlatformCompatArtifact(AndroidArtifact):
"""
Parser for uninstalled apps listed in platform_compat section.
"""
def check_indicators(self) -> None:
if not self.indicators:
return
for result in self.results:
ioc = self.indicators.check_app_id(result["package_name"])
if ioc:
result["matched_indicator"] = ioc
self.detected.append(result)
continue
def parse(self, data: str) -> None:
for line in data.splitlines():
if not line.startswith("ChangeId(168419799; name=DOWNSCALED;"):
continue
if line.strip() == "":
break
# Look for rawOverrides field
if "rawOverrides={" in line:
# Extract the content inside the braces for rawOverrides
overrides_field = line.split("rawOverrides={", 1)[1].split("};", 1)[0]
for entry in overrides_field.split(", "):
# Extract app name
uninstall_app = entry.split("=")[0].strip()
self.results.append({"package_name": uninstall_app})

View File

@@ -0,0 +1,43 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
from typing import Union
from .artifact import AndroidArtifact
class FileTimestampsArtifact(AndroidArtifact):
def serialize(self, record: dict) -> Union[dict, list]:
records = []
for ts in set(
[
record.get("access_time"),
record.get("changed_time"),
record.get("modified_time"),
]
):
if not ts:
continue
macb = ""
macb += "M" if ts == record.get("modified_time") else "-"
macb += "A" if ts == record.get("access_time") else "-"
macb += "C" if ts == record.get("changed_time") else "-"
macb += "-"
msg = record["path"]
if record.get("context"):
msg += f" ({record['context']})"
records.append(
{
"timestamp": ts,
"module": self.__class__.__name__,
"event": macb,
"data": msg,
}
)
return records

View File

@@ -42,6 +42,17 @@ class GetProp(AndroidArtifact):
entry = {"name": matches[0][0], "value": matches[0][1]}
self.results.append(entry)
def get_device_timezone(self) -> str:
"""
Get the device timezone from the getprop results
Used in other moduels to calculate the timezone offset
"""
for entry in self.results:
if entry["name"] == "persist.sys.timezone":
return entry["value"]
return None
def check_indicators(self) -> None:
for entry in self.results:
if entry["name"] in INTERESTING_PROPERTIES:

View File

@@ -16,6 +16,11 @@ ANDROID_DANGEROUS_SETTINGS = [
"key": "package_verifier_enable",
"safe_value": "1",
},
{
"description": "disabled APK package verification",
"key": "package_verifier_state",
"safe_value": "1",
},
{
"description": "disabled Google Play Protect",
"key": "package_verifier_user_consent",

File diff suppressed because it is too large Load Diff

View File

@@ -326,8 +326,7 @@ class AndroidExtraction(MVTModule):
if not header["backup"]:
self.log.error(
"Extracting SMS via Android backup failed. "
"No valid backup data found."
"Extracting SMS via Android backup failed. No valid backup data found."
)
return None

View File

@@ -75,8 +75,7 @@ class Packages(AndroidExtraction):
for result in self.results:
if result["package_name"] in ROOT_PACKAGES:
self.log.warning(
"Found an installed package related to "
'rooting/jailbreaking: "%s"',
'Found an installed package related to rooting/jailbreaking: "%s"',
result["package_name"],
)
self.detected.append(result)

View File

@@ -70,7 +70,7 @@ class SMS(AndroidExtraction):
"timestamp": record["isodate"],
"module": self.__class__.__name__,
"event": f"sms_{record['direction']}",
"data": f"{record.get('address', 'unknown source')}: \"{body}\"",
"data": f'{record.get("address", "unknown source")}: "{body}"',
}
def check_indicators(self) -> None:

View File

@@ -14,6 +14,7 @@ from .dumpsys_receivers import DumpsysReceivers
from .dumpsys_adb import DumpsysADBState
from .getprop import Getprop
from .packages import Packages
from .dumpsys_platform_compat import DumpsysPlatformCompat
from .processes import Processes
from .settings import Settings
from .sms import SMS
@@ -29,6 +30,7 @@ ANDROIDQF_MODULES = [
DumpsysBatteryHistory,
DumpsysADBState,
Packages,
DumpsysPlatformCompat,
Processes,
Getprop,
Settings,

View File

@@ -48,6 +48,37 @@ class AndroidQFModule(MVTModule):
def _get_files_by_pattern(self, pattern: str):
return fnmatch.filter(self.files, pattern)
def _get_device_timezone(self):
"""
Get the device timezone from the getprop.txt file.
This is needed to map local timestamps stored in some
Android log files to UTC/timezone-aware timestamps.
"""
get_prop_files = self._get_files_by_pattern("*/getprop.txt")
if not get_prop_files:
self.log.warning(
"Could not find getprop.txt file. "
"Some timestamps and timeline data may be incorrect."
)
return None
from mvt.android.artifacts.getprop import GetProp
properties_artifact = GetProp()
prop_data = self._get_file_content(get_prop_files[0]).decode("utf-8")
properties_artifact.parse(prop_data)
timezone = properties_artifact.get_device_timezone()
if timezone:
self.log.debug("Identified local phone timezone: %s", timezone)
return timezone
self.log.warning(
"Could not find or determine local device timezone. "
"Some timestamps and timeline data may be incorrect."
)
return None
def _get_file_content(self, file_path):
if self.archive:
handle = self.archive.open(file_path)

View File

@@ -0,0 +1,44 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
from typing import Optional
from mvt.android.artifacts.dumpsys_platform_compat import DumpsysPlatformCompatArtifact
from .base import AndroidQFModule
class DumpsysPlatformCompat(DumpsysPlatformCompatArtifact, AndroidQFModule):
"""This module extracts details on uninstalled apps."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: Optional[list] = None,
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
def run(self) -> None:
dumpsys_file = self._get_files_by_pattern("*/dumpsys.txt")
if not dumpsys_file:
return
data = self._get_file_content(dumpsys_file[0]).decode("utf-8", errors="replace")
content = self.extract_dumpsys_section(data, "DUMP OF SERVICE platform_compat:")
self.parse(content)
self.log.info("Found %d uninstalled apps", len(self.results))

View File

@@ -6,6 +6,11 @@
import datetime
import json
import logging
try:
import zoneinfo
except ImportError:
from backports import zoneinfo
from typing import Optional, Union
from mvt.android.modules.androidqf.base import AndroidQFModule
@@ -74,7 +79,7 @@ class Files(AndroidQFModule):
for result in self.results:
ioc = self.indicators.check_file_path(result["path"])
if ioc:
result["matched_indicator"] == ioc
result["matched_indicator"] = ioc
self.detected.append(result)
continue
@@ -106,6 +111,12 @@ class Files(AndroidQFModule):
# TODO: adds SHA1 and MD5 when available in MVT
def run(self) -> None:
if timezone := self._get_device_timezone():
device_timezone = zoneinfo.ZoneInfo(timezone)
else:
self.log.warning("Unable to determine device timezone, using UTC")
device_timezone = zoneinfo.ZoneInfo("UTC")
for file in self._get_files_by_pattern("*/files.json"):
rawdata = self._get_file_content(file).decode("utf-8", errors="ignore")
try:
@@ -120,11 +131,18 @@ class Files(AndroidQFModule):
for file_data in data:
for ts in ["access_time", "changed_time", "modified_time"]:
if ts in file_data:
file_data[ts] = convert_datetime_to_iso(
datetime.datetime.fromtimestamp(
file_data[ts], tz=datetime.timezone.utc
)
utc_timestamp = datetime.datetime.fromtimestamp(
file_data[ts], tz=datetime.timezone.utc
)
# Convert the UTC timestamp to local tiem on Android device's local timezone
local_timestamp = utc_timestamp.astimezone(device_timezone)
# HACK: We only output the UTC timestamp in convert_datetime_to_iso, we
# set the timestamp timezone to UTC, to avoid the timezone conversion again.
local_timestamp = local_timestamp.replace(
tzinfo=datetime.timezone.utc
)
file_data[ts] = convert_datetime_to_iso(local_timestamp)
self.results.append(file_data)

View File

@@ -0,0 +1,65 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import os
import datetime
import logging
from typing import Optional
from mvt.common.utils import convert_datetime_to_iso
from .base import AndroidQFModule
from mvt.android.artifacts.file_timestamps import FileTimestampsArtifact
class LogsFileTimestamps(FileTimestampsArtifact, AndroidQFModule):
"""This module extracts records from battery daily updates."""
slug = "logfile_timestamps"
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: Optional[list] = None,
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
def _get_file_modification_time(self, file_path: str) -> dict:
if self.archive:
file_timetuple = self.archive.getinfo(file_path).date_time
return datetime.datetime(*file_timetuple)
else:
file_stat = os.stat(os.path.join(self.parent_path, file_path))
return datetime.datetime.fromtimestamp(file_stat.st_mtime)
def run(self) -> None:
filesystem_files = self._get_files_by_pattern("*/logs/*")
self.results = []
for file in filesystem_files:
# Only the modification time is available in the zip file metadata.
# The timezone is the local timezone of the machine the phone.
modification_time = self._get_file_modification_time(file)
self.results.append(
{
"path": file,
"modified_time": convert_datetime_to_iso(modification_time),
}
)
self.log.info(
"Extracted a total of %d filesystem timestamps from AndroidQF logs directory.",
len(self.results),
)

View File

@@ -44,8 +44,7 @@ class Packages(AndroidQFModule):
for result in self.results:
if result["name"] in ROOT_PACKAGES:
self.log.warning(
"Found an installed package related to "
'rooting/jailbreaking: "%s"',
'Found an installed package related to rooting/jailbreaking: "%s"',
result["name"],
)
self.detected.append(result)

View File

@@ -3,10 +3,11 @@
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import os
from rich.prompt import Prompt
from mvt.common.config import settings
MVT_ANDROID_BACKUP_PASSWORD = "MVT_ANDROID_BACKUP_PASSWORD"
@@ -16,24 +17,24 @@ def cli_load_android_backup_password(log, backup_password):
Used in MVT CLI command parsers.
"""
password_from_env = os.environ.get(MVT_ANDROID_BACKUP_PASSWORD, None)
password_from_env_or_config = settings.ANDROID_BACKUP_PASSWORD
if backup_password:
log.info(
"Your password may be visible in the process table because it "
"was supplied on the command line!"
)
if password_from_env:
if password_from_env_or_config:
log.info(
"Ignoring %s environment variable, using --backup-password argument instead",
MVT_ANDROID_BACKUP_PASSWORD,
"MVT_ANDROID_BACKUP_PASSWORD",
)
return backup_password
elif password_from_env:
elif password_from_env_or_config:
log.info(
"Using backup password from %s environment variable",
MVT_ANDROID_BACKUP_PASSWORD,
"Using backup password from %s environment variable or config file",
"MVT_ANDROID_BACKUP_PASSWORD",
)
return password_from_env
return password_from_env_or_config
def prompt_or_load_android_backup_password(log, module_options):

View File

@@ -11,8 +11,11 @@ from .battery_history import BatteryHistory
from .dbinfo import DBInfo
from .getprop import Getprop
from .packages import Packages
from .platform_compat import PlatformCompat
from .receivers import Receivers
from .adb_state import DumpsysADBState
from .fs_timestamps import BugReportTimestamps
from .tombstones import Tombstones
BUGREPORT_MODULES = [
Accessibility,
@@ -23,6 +26,9 @@ BUGREPORT_MODULES = [
DBInfo,
Getprop,
Packages,
PlatformCompat,
Receivers,
DumpsysADBState,
BugReportTimestamps,
Tombstones,
]

View File

@@ -2,10 +2,11 @@
# Copyright (c) 2021-2023 The MVT Authors.
# See the file 'LICENSE' for usage and copying permissions, or find a copy at
# https://github.com/mvt-project/mvt/blob/main/LICENSE
import datetime
import fnmatch
import logging
import os
from typing import List, Optional
from zipfile import ZipFile
@@ -91,3 +92,11 @@ class BugReportModule(MVTModule):
return None
return self._get_file_content(dumpstate_logs[0])
def _get_file_modification_time(self, file_path: str) -> dict:
if self.zip_archive:
file_timetuple = self.zip_archive.getinfo(file_path).date_time
return datetime.datetime(*file_timetuple)
else:
file_stat = os.stat(os.path.join(self.extract_path, file_path))
return datetime.datetime.fromtimestamp(file_stat.st_mtime)

View File

@@ -0,0 +1,55 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
from typing import Optional
from mvt.common.utils import convert_datetime_to_iso
from .base import BugReportModule
from mvt.android.artifacts.file_timestamps import FileTimestampsArtifact
class BugReportTimestamps(FileTimestampsArtifact, BugReportModule):
"""This module extracts records from battery daily updates."""
slug = "bugreport_timestamps"
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: Optional[list] = None,
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
def run(self) -> None:
filesystem_files = self._get_files_by_pattern("FS/*")
self.results = []
for file in filesystem_files:
# Only the modification time is available in the zip file metadata.
# The timezone is the local timezone of the machine the phone.
modification_time = self._get_file_modification_time(file)
self.results.append(
{
"path": file,
"modified_time": convert_datetime_to_iso(modification_time),
}
)
self.log.info(
"Extracted a total of %d filesystem timestamps from bugreport.",
len(self.results),
)

View File

@@ -0,0 +1,48 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
from typing import Optional
from mvt.android.artifacts.dumpsys_platform_compat import DumpsysPlatformCompatArtifact
from mvt.android.modules.bugreport.base import BugReportModule
class PlatformCompat(DumpsysPlatformCompatArtifact, BugReportModule):
"""This module extracts details on uninstalled apps."""
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: Optional[list] = None,
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
def run(self) -> None:
data = self._get_dumpstate_file()
if not data:
self.log.error(
"Unable to find dumpstate file. "
"Did you provide a valid bug report archive?"
)
return
data = data.decode("utf-8", errors="replace")
content = self.extract_dumpsys_section(data, "DUMP OF SERVICE platform_compat:")
self.parse(content)
self.log.info("Found %d uninstalled apps", len(self.results))

View File

@@ -0,0 +1,64 @@
# Mobile Verification Toolkit (MVT)
# Copyright (c) 2021-2023 The MVT Authors.
# Use of this software is governed by the MVT License 1.1 that can be found at
# https://license.mvt.re/1.1/
import logging
from typing import Optional
from mvt.android.artifacts.tombstone_crashes import TombstoneCrashArtifact
from .base import BugReportModule
class Tombstones(TombstoneCrashArtifact, BugReportModule):
"""This module extracts records from battery daily updates."""
slug = "tombstones"
def __init__(
self,
file_path: Optional[str] = None,
target_path: Optional[str] = None,
results_path: Optional[str] = None,
module_options: Optional[dict] = None,
log: logging.Logger = logging.getLogger(__name__),
results: Optional[list] = None,
) -> None:
super().__init__(
file_path=file_path,
target_path=target_path,
results_path=results_path,
module_options=module_options,
log=log,
results=results,
)
def run(self) -> None:
tombstone_files = self._get_files_by_pattern("*/tombstone_*")
if not tombstone_files:
self.log.error(
"Unable to find any tombstone files. "
"Did you provide a valid bugreport archive?"
)
return
for tombstone_file in sorted(tombstone_files):
tombstone_filename = tombstone_file.split("/")[-1]
modification_time = self._get_file_modification_time(tombstone_file)
tombstone_data = self._get_file_content(tombstone_file)
try:
if tombstone_file.endswith(".pb"):
self.parse_protobuf(
tombstone_filename, modification_time, tombstone_data
)
else:
self.parse(tombstone_filename, modification_time, tombstone_data)
except ValueError as e:
# Catch any exceptions raised during parsing or validation.
self.log.error(f"Error parsing tombstone file {tombstone_file}: {e}")
self.log.info(
"Extracted a total of %d tombstone files",
len(self.results),
)

Some files were not shown because too many files have changed in this diff Show More