github
/
bitcoin
mirror of https://github.com/bitcoin/bitcoin
1
Fork 0
Code Releases Activity
40,787 Commits 5 Branches 309 Tags 4.8 GiB
C++ 64.2%
Python 19.9%
C 11.8%
M4 1.2%
Shell 1%
Other 1.8%
Go to file
Ava Chow 1ffbd96349
Merge bitcoin/bitcoin#29771: test: Run framework unit tests in parallel 
f19f0a2e5a test: Run framework unit tests in parallel (tdb3)

Pull request description:

  Functional test framework unit tests are currently run prior to all other functional tests.

  This PR enables execution of the test framework unit tests in parallel with the functional tests, rather than before the functional tests, saving runtime and more efficiently using available cores.

  This is a follow up to  https://github.com/bitcoin/bitcoin/pull/29470#issuecomment-1962313977

  ### New behavior:
  1) When running all tests, the framework unit tests are run in parallel with the other tests (unless explicitly skipped with `--exclude`).  This parallelization introduces marginal time savings when running all tests, depending on the machine used.  As an example, a 2-3% time savings (9 seconds) was observed on a machine using `--jobs=18` (with 18 available cores).
  2) When running specific functional tests, framework unit tests are now skipped by default.  Framework unit tests can be added by including `feature_framework_unit_tests.py` in the list of specific tests being executed.  The rationale for skipping by default is that if the tester is running specific functional tests, there is a conscious decision to focus testing, and choosing to run all tests (where unit tests are run by default) would be a next step.
  3) The `--skipunit` option is now removed since unit tests are parallelized (they no longer delay other tests).  Unit tests are treated equally as functional tests.

  ### Implementation notes:
  Since `TextTestRunner` can be noisy (even with verbosity=0, and therefore trigger job failure through the presence of non-failure stderr output), the approach taken was to send output to stdout, and forward test result (as determined by `TestResult` returned).  This aligns with the previous check for unit test failure (`if not result.wasSuccessful():`).

  This approach was tested by inserting `self.assertEquals(True, False)` into test_framework/address.py and seeing specifics of the failure reported.

  ```
  135/302 - feature_framework_unit_tests.py failed, Duration: 0 s

  stdout:
  .F
  ======================================================================
  FAIL: test_bech32_decode (test_framework.address.TestFrameworkScript.test_bech32_decode)
  ----------------------------------------------------------------------
  Traceback (most recent call last):
    File "/home/dev/myrepos/bitcoin/test/functional/test_framework/address.py", line 228, in test_bech32_decode
      self.assertEqual(True, False)
  AssertionError: True != False

  ----------------------------------------------------------------------
  Ran 2 tests in 0.003s

  FAILED (failures=1)

  stderr:
  ```

  There was an initial thought to parallelize the execution of the unit tests themselves (i.e. run the 12 unit test files in parallel), however, this is not anticipated to further reduce runtime meaningfully and is anticipated to add unnecessary complexity.

ACKs for top commit:
  maflcko:
    ACK f19f0a2e5a 🌽
  achow101:
    ACK f19f0a2e5a
  kevkevinpal:
    Approach ACK f19f0a2e5a

Tree-SHA512: ab9f82c30371b2242bc7a263ea0e25d35e68e2ddf223d2a55498ad940d1e5b73bba76cce8b264d71e2ed31b753430d8ef8d57efc1e4fd9ced7fb845e27f4f47e
 		2024-04-26 16:06:37 -04:00
.github ci: Pull in qtbase5-dev instead of seperate low-level libraries 2024-03-29 11:17:39 +01:00
.tx qt: Bump Transifex slug for 27.x 2024-02-07 09:24:32 +00:00
build-aux/m4 build: Fix false positive `CHECK_ATOMIC` test for clang-15 2024-04-12 09:52:06 +02:00
build_msvc remove libbitcoinconsensus 2024-03-18 16:59:39 +00:00
ci Merge bitcoin/bitcoin#29837: ci: disable `_FORTIFY_SOURCE` with MSAN 2024-04-24 22:46:25 +08:00
contrib Merge bitcoin/bitcoin#29890: deploy: remove some tools when cross-compiling for macOS 2024-04-25 21:21:42 +08:00
depends Merge bitcoin/bitcoin#29953: doc: Bash is needed in gen_id and is not installed on FreeBSD by default 2024-04-25 21:22:45 +08:00
doc Merge bitcoin/bitcoin#29708: depends: build libnatpmp with CMake 2024-04-25 21:07:52 +08:00
share contrib: rpcauth.py - Add new option (-j/--json) to output text in json format 2024-04-25 08:32:28 -05:00
src test: Add two more urlDecode tests 2024-04-26 08:33:56 +02:00
test Merge bitcoin/bitcoin#29771: test: Run framework unit tests in parallel 2024-04-26 16:06:37 -04:00
.cirrus.yml ci: Skip git install if it is already installed 2024-02-16 16:06:45 +01:00
.editorconfig ci: Drop AppVeyor CI integration 2021-09-07 06:12:53 +03:00
.gitattributes Separate protocol versioning from clientversion 2014-10-29 00:24:40 -04:00
.gitignore Merge bitcoin/bitcoin#29733: build, macos: Drop unused `osx_volname` target 2024-04-02 14:57:22 +01:00
.python-version Bump .python-version from 3.9.17 to 3.9.18 2023-10-24 18:51:24 +02:00
.style.yapf Update .style.yapf 2023-06-01 23:35:10 +05:30
CONTRIBUTING.md Update the developer mailing list address. 2024-04-02 15:38:40 -03:00
COPYING doc: upgrade Bitcoin Core license to 2024 2024-01-10 16:29:01 -06:00
INSTALL.md doc: Added hyperlink for doc/build 2021-09-09 19:53:12 +05:30
Makefile.am build: don't use install_name_tool for macOS deploy when cross-compiling 2024-04-17 11:40:25 +01:00
README.md doc: Explain Bitcoin Core in README.md 2022-05-10 07:49:09 +02:00
SECURITY.md Update security.md contact for achow101 2023-12-14 18:14:54 -05:00
autogen.sh build: make sure we can overwrite config.{guess,sub} 2023-06-13 14:58:43 +02:00
configure.ac Merge bitcoin/bitcoin#29904: refactor: Use our own implementation of urlDecode 2024-04-25 13:02:43 -04:00

README.md

Bitcoin Core integration/staging tree

https://bitcoincore.org

For an immediately usable, binary version of the Bitcoin Core software, see https://bitcoincore.org/en/download/.

What is Bitcoin Core?

Bitcoin Core connects to the Bitcoin peer-to-peer network to download and fully validate blocks and transactions. It also includes a wallet and graphical user interface, which can be optionally built.

Further information about Bitcoin Core is available in the doc folder.

License

Bitcoin Core is released under the terms of the MIT license. See COPYING for more information or see https://opensource.org/licenses/MIT.

Development Process

The master branch is regularly built (see doc/build-*.md for instructions) and tested, but it is not guaranteed to be completely stable. Tags are created regularly from release branches to indicate new official, stable release versions of Bitcoin Core.

The https://github.com/bitcoin-core/gui repository is used exclusively for the development of the GUI. Its master branch is identical in all monotree repositories. Release branches and tags do not exist, so please do not fork that repository unless it is for development reasons.

The contribution workflow is described in CONTRIBUTING.md and useful hints for developers can be found in doc/developer-notes.md.

Testing

Testing and code review is the bottleneck for development; we get more pull requests than we can review and test on short notice. Please be patient and help out by testing other people's pull requests, and remember this is a security-critical project where any mistake might cost people lots of money.

Automated Testing

Developers are strongly encouraged to write unit tests for new code, and to submit new unit tests for old code. Unit tests can be compiled and run (assuming they weren't disabled in configure) with: make check. Further details on running and extending unit tests can be found in /src/test/README.md.

There are also regression and integration tests, written in Python. These tests can be run (if the test dependencies are installed) with: test/functional/test_runner.py

The CI (Continuous Integration) systems make sure that every pull request is built for Windows, Linux, and macOS, and that unit/sanity tests are run automatically.

Manual Quality Assurance (QA) Testing

Changes should be tested by somebody other than the developer who wrote the code. This is especially important for large or high-risk changes. It is useful to add a test plan to the pull request description if testing the changes is not straightforward.

Translations

Changes to translations as well as new translations can be submitted to Bitcoin Core's Transifex page.

Translations are periodically pulled from Transifex and merged into the git repository. See the translation process for details on how this works.

Important: We do not accept translation changes as GitHub pull requests because the next pull from Transifex would automatically overwrite them again.