Commit Graph

139 Commits

Author SHA1 Message Date
Hennadii Stepanov 3d31abbaaa
build: Make Windows-specific targets available for Windows builds only 2021-03-18 20:58:13 +02:00
Hennadii Stepanov 92990e25b7
build: Make macOS-specific targets available for macOS builds only 2021-03-18 20:58:04 +02:00
Carl Dong b3bdff42b5 build: Proper quoting for var printing targets
Previously, if the value contained syntax that was meaningful to make,
the printing would fail. Quoting properly avoids this.
2021-02-03 12:10:02 -05:00
MarcoFalke faff3991a9
ci: Fuzz with integer sanitizer 2021-01-26 12:50:43 +01:00
Carl Dong f3835dc6a3 build: Make xorrisofs reproducible with -volume_date
We need this to be after a '--' as '-volume_date' is a xorriso
flag, not a xorrisofs flag. See the respective man pages.

For more details: https://issues.guix.info/issue/35283#2
2021-01-21 10:58:12 -05:00
Wladimir J. van der Laan 7b975639ef
Merge #19937: signet mining utility
595a34dbea contrib/signet: Document miner script in README.md (Anthony Towns)
ff7dbdc08a contrib/signet: Add script for generating a signet chain (Anthony Towns)
13762bcc96 Add bitcoin-util command line utility (Anthony Towns)
95d5d5e625 rpc: allow getblocktemplate for test chains when unconnected or in IBD (Anthony Towns)
81c54dec20 rpc: update getblocktemplate with signet rule, include signet_challenge (Anthony Towns)

Pull request description:

  Adds `contrib/signet/miner` for mining signet blocks.

  Adds `bitcoin-util` cli utility, with the idea being it can provide bitcoin related functionality that does not rely on the ability to access a running node. Only subcommand currently is "grind" which takes a hex-encoded header and grinds its nonce until its nBits is satisfied.

  Updates `getblocktemplate` to include `signet_challenge` field, and makes `getblocktemplate` require the signet rule when invoked on the signet change. Removes connectivity and IBD checks from `getblocktemplate` when applied to a test chain (regtest, testnet, signet).

ACKs for top commit:
  laanwj:
    code review ACK 595a34dbea

Tree-SHA512: 8d43297710fdc1edc58acd9b53e1bd1671e5724f7097b40ab73653715dc8becc70534c4496cbba9290f4dd6538a7a3d5830eb85f83391ea31a3bb5b9d3378cc3
2021-01-12 12:53:45 +01:00
Anthony Towns 13762bcc96 Add bitcoin-util command line utility 2021-01-12 18:34:25 +10:00
fanquake ddbf7a6f50
Merge #20684: build: Define .INTERMEDIATE target once only
5e0dedb111 build: Define .INTERMEDIATE target once only (Hennadii Stepanov)

Pull request description:

  A new warning was introduced in 22437fc72e (#20470):
  ```
  $ ./autogen.sh
  ...
  Makefile.am:335: warning: .INTERMEDIATE was already defined in condition !BUILD_DARWIN, which is included in condition TRUE ...
  Makefile.am:139: ... '.INTERMEDIATE' previously defined here
  ...
  ```

  Fixed in this PR.

ACKs for top commit:
  jonatack:
    Tested and very light review ACK 5e0dedb111

Tree-SHA512: ecf8de79ba394c36ee84e0b8d3ba78587e0f856259e9731e6bbb38d0baebfd083eb44d7ef6a386dd9e4508dd64fec1c2b9a007e175fbd4d986e845b1c300a649
2020-12-28 16:48:06 +08:00
Wladimir J. van der Laan f1dbf92ff0
Merge #20434: contrib: Parse ELF directly for symbol and security checks
a0a771843f contrib: Changes to checks for PowerPC64 (Luke Dashjr)
634f6ec4eb contrib: Parse ELF directly for symbol and security checks (Wladimir J. van der Laan)

Pull request description:

  Instead of the ever-messier text parsing of the output of the readelf tool (which is clearly meant for human consumption not to be machine parseable), parse the ELF binaries directly.

  Add a small dependency-less ELF parser specific to the checks.

  This is slightly more secure, too, because it removes potential ambiguity due to misparsing and changes in the output format of `elfread`. It also allows for stricter and more specific ELF format checks in the future.

  This removes the build-time dependency for `readelf`.

  It passes the test-security-check for me locally, ~~though I haven't checked on all platforms~~. I've checked that this works on the cross-compile output for all ELF platforms supported by Bitcoin Core at the moment, as well as PPC64 LE and BE.

Top commit has no ACKs.

Tree-SHA512: 7f9241fec83ee512642fecf5afd90546964561efd8c8c0f99826dcf6660604a4db2b7255e1afb1e9bb0211fd06f5dbad18a6175dfc03e39761a40025118e7bfc
2020-12-18 12:16:40 +01:00
Hennadii Stepanov 5e0dedb111
build: Define .INTERMEDIATE target once only 2020-12-17 14:10:52 +02:00
fanquake 0df9819126 build: Replace genisoimage with xorriso
xorriso and its mkisofs/genisoimage emulation alter-ego xorrisofs are
more maintained, and has the right toggles for us to achieve output
determinism without using blunt tools like faketime.

In this commit, we use xorrisofs from the build environment rather than
building it ourselves using depends. This is not necessary and can be
changed in the future.

From https://wiki.debian.org/genisoimage?action=recall&rev=11 :

> The classical command line interface for production of ISO 9660
> filesystem images is the option set established by program mkisofs.
> For reasons of licensing and other problems with its author, Debian
> ships a fork of mkisofs, called genisoimage, which was split off in
> 2006 and then developed independently.
>
> Meanwhile, genisoimage gets no new features and not even bug fixes. It
> is first choice only if its options -udf or -hfs are needed.
>
> Replacement in most uses cases, especially for bootable ISO 9660
> filesystems, archiving, and backup, is xorrisofs which starts the -as
> mkisofs emulation mode of program xorriso.
2020-12-16 12:29:14 -05:00
Carl Dong 22437fc72e build: Run libdmg-hfsplus's DMG tool in make deploy
Previously, the compression of the .iso file to a .dmg file was done
outside of `make deploy' in order to use the faketime-wrapped version of
libdmg-hfsplus's DMG tool.

Specifying the faketime-wrapped version of the DMG tool to ./configure
fixes this and simplifies build scripts.
2020-12-11 11:43:11 -05:00
Wladimir J. van der Laan 8e1f9d6273
Merge #20549: Support make src/bitcoin-node and src/bitcoin-gui
206f74e88c Support make src/bitcoin-node and src/bitcoin-gui (João Barbosa)

Pull request description:

  This change adds the following configure output variables
  ```
  dnl Multi Process
  BITCOIN_MP_NODE_NAME=bitcoin-node
  BITCOIN_MP_GUI_NAME=bitcoin-gui
  ```
  and adds support for
  ```sh
  make src/bitcoin-node src/bitcoin-gui
  ```

ACKs for top commit:
  laanwj:
    Code review ACK 206f74e88c

Tree-SHA512: 4d1a694b9010ecc267ee955f4475127a58e6da72f30179ec740285ee6fe03cd91dcb6847317a47460dbd548edb88b7da6c7a98eac10f0dabe3ce4e83e0aa8093
2020-12-10 12:53:54 +01:00
fanquake ae9b48995b
contrib: add symbol check test for PE 2020-12-09 23:13:24 +08:00
fanquake 16b31cc4c5
Merge #20422: build: mac deployment unification
b685f60a08 build: mac_alias 2.1.1 (fanquake)
5d2cbdf772 macdeploy: use Python 3.6 (fanquake)
a42aa94c54 macdeploy: remove runHDIUtil in favor of directly calling subprocess.run (fanquake)
adaa26202b macdeploy: remove existing Bitcoin-Core.dmg if present (fanquake)
ccb0325b1b macdeploy: move qt_conf to where it's used (fanquake)
6390a04862 macdeploy: consolidate .DS_Store generation (fanquake)
32347cd56a macdeploy: assume plistlib is available (fanquake)
0ab4018c12 macdeploy: have a single level of logging output (fanquake)
827d382aa7 macdeploy: remove add-resources argument (fanquake)
464b34d4c3 macdeploy: remove codesigning argument (fanquake)
4d70d3d7fe build: automatically determine macOS translations (fanquake)

Pull request description:

  This consolidates our macOS build code so that `.DS_Store` generation is the same when running `make deploy` for macOS when building on Linux and macOS, rather than maintaining two version of code that essentially do the same thing (just slightly differently).

  It also removes unused code and any AppleScript usage, automates finding translation files and generally simplifies `macdeployqtplus`. It also gets rid of the annoying "popping up" behaviour during DMG generation, names the created image `Bitcoin-Core.dmg` rather than `Bitcoin-Qt.dmg`.

ACKs for top commit:
  dergoegge:
    ACK b685f60a08 - Less and cleaner code looks good. I tested this with `make deploy` and everything still works + the popup during DMG generation is gone.

Tree-SHA512: dcd38344e2dfcfa7ffbccf6226a71425c4d16b421a4881d5ee37b8e7ef393b3e8077262444c39b11912269d8cf688aba897e6518cba8361eb24a03fdd03b8caf
2020-12-08 16:51:49 +08:00
fanquake ed1bbcefea contrib: add MACHO tests to symbol-check tests 2020-12-03 12:16:19 +01:00
Wladimir J. van der Laan 5bab08df17 contrib: Add test for ELF symbol-check
Check both failure cases:
- Use a glibc symbol from a version that is too new
- Use a symbol from a library that is not in the allowlist

And also check a conforming binary.

Adding a similar check for Windows PE can be done in a separate PR.
2020-12-03 12:15:12 +01:00
João Barbosa 206f74e88c Support make src/bitcoin-node and src/bitcoin-gui 2020-12-02 23:05:35 +00:00
fanquake 6390a04862
macdeploy: consolidate .DS_Store generation
Rather than two lots of logic doing roughly the same thing, dependent on if
you're compiling on Linux or macOS, combine the .DS store generation into
macdeployqtplus.

This also removes the -fancy and -volname options.
2020-11-30 14:54:19 +08:00
fanquake 0ab4018c12
macdeploy: have a single level of logging output
4 different levels of verbosity is overkill for a fairly simple script, which
was always being run at 2 in any case.
2020-11-30 14:54:19 +08:00
fanquake 4d70d3d7fe
build: automatically determine macOS translations
Rather than using OSX_QT_TRANSLATIONS which must be manually updated,
and we forget to update anyway, i.e: #19059, automatically find and copy
available translations from the translations directory.
2020-11-30 14:54:18 +08:00
Wladimir J. van der Laan 634f6ec4eb contrib: Parse ELF directly for symbol and security checks
Instead of the ever-messier text parsing of the output of the readelf
tool (which is clearly meant for human consumption not to be machine
parseable), parse the ELF binaries directly.

Add a small dependency-less ELF parser specific to the checks.

This is slightly more secure, too, because it removes potential
ambiguity due to misparsing and changes in the output format of `elfread`. It
also allows for stricter and more specific ELF format checks in the future.

This removes the build-time dependency for `readelf`.

It passes the test-security-check for me locally, though I haven't
checked on all platforms.
2020-11-22 11:11:32 +01:00
eugene a9f201439a build: use DIR_FUZZ_SEED_CORPUS if specified for cov_fuzz target
This commit allows the user to specify the location of the
fuzz_seed_corpus directory on their machine when running the cov_fuzz
target. If DIR_FUZZ_SEED_CORPUS is specified, then this will be used.
Otherwise, qa-assets/fuzz_seed_corpus is assumed to be in the bitcoin
directory.
2020-09-08 02:45:42 -04:00
eugene 9bdde3c802 build: add /usr/local/ to LCOV_FILTER_PATTERN for macOS builds
With this commit, the files in /usr/local/ will not be included in
`make cov` or `make cov_fuzz` coverage reports. This behavior could
be observed when generating the reports on macOS with brew-installed
clang.
2020-09-02 15:17:13 -04:00
Hennadii Stepanov 8ebc0505e9
build: Add missed fuzz_filtered.info to COVERAGE_INFO 2020-08-08 23:38:14 +03:00
Hennadii Stepanov c71bdf93d7
build, test: Add support for llvm-cov 2020-08-08 22:53:15 +03:00
fanquake 968aaae940
tests: run test-security-check.py in CI 2020-06-16 19:52:30 +08:00
fanquake 69bfcac27a
gui: update Qt base translations for macOS release
These haven't been updated since their addition, so this updates the list that
controls which qt base translations are bundled with the macOS binary, to all the
languages that are available with qt 5.9.8.

This could probably be improved in some way, however qt updates are infrequent,
and I didn't want to spend any more time looking at this. Also given that no-one
seems to have noticed and/or reported this it wouldn't seem high-priority.

Could be backported to 0.20.1.
2020-05-23 18:22:29 +08:00
Carl Dong 27e63e01cc
build: Accomodate makensis v2.x
Apparently the -X flag doesn't work as expected in makensis v2.x

For example:

  makensis -V2 share/setup.nsi -X'OutFile "test.exe"'

Will output:

  OutFile expects 1 parameters, got 0.
  Usage: OutFile install_output.exe

So let's instead construct the file using POSIX-compliant commands and
shell constructs
2020-05-01 14:27:57 -04:00
Carl Dong fa791da02f
nsis: Specify OutFile path only once
Previously, we would specify the makensis output file path twice:

  1. At the top of Makefile.am as BITCOIN_WIN_INSTALLER, and
  2. In share/setup.nsi.ini

This commit uses the -X flag of makensis to eliminate the need for the
second instance mentioned above, referring makensis directly to the
value of BITCOIN_WIN_INSTALLER
2020-04-28 10:36:38 -04:00
Wladimir J. van der Laan bdc2644b72
Merge #18107: build: Add cov_fuzz target
faf7d4fa86 build: Add cov_fuzz target (MarcoFalke)
fac71e364e build: link fuzz/test_runner.py for out-of-tree builds (MarcoFalke)
faf2c5aca0 build: Remove unused USE_COVERAGE (MarcoFalke)

Pull request description:

  Only libFuzzer is supported right now, so clang is required. Thus, this needs a workaround such as https://github.com/bitcoin/bitcoin/issues/12602#issuecomment-562788247

  Can be tested with:

  ```
  mkdir build && cd build
  ../configure --enable-fuzz --with-sanitizers=fuzzer --enable-lcov --enable-lcov-branch-coverage CC=clang CXX=clang++
  make $MAKEJOBS
  make cov_fuzz

ACKs for top commit:
  practicalswift:
    ACK faf7d4fa86

Tree-SHA512: 6828f8f81d95f6781713d0b09d7eba2ffdb50217e09ca839db61791a4ed70024859c7a0cb01d9eede79166d574dd57ece01f9d9fe2610d4a72a4ca4a4ce0b838
2020-03-27 14:29:48 +01:00
Wladimir J. van der Laan b53af72b82
Merge #18416: util: Limit decimal range of numbers ParseScript accepts
9ab14e4d21 Limit decimal range of numbers ParseScript accepts (pierrenn)

Pull request description:

  Following up on this suggestion : https://github.com/bitcoin/bitcoin/pull/18413#issuecomment-602966490, prevent the output of `atoi64` in the `core_read.cpp:ParseScript` helper to send to `CScriptNum::serialize` values wider than 32-bit.

  Since the `ParseScript` helper is only used by the tool defined in `bitcoin-tx.cpp`, this only prevents users to provide too much unrealistic values.

ACKs for top commit:
  laanwj:
    ACK 9ab14e4d21

Tree-SHA512: ee228269d19d04e8fee0aa7c0ae2bb0a2b437b8e574356e8d9b2279318242057d51fcf39a842aa3afe27408d0f2d5276df245d07a3f4828644a366f80587b666
2020-03-27 08:02:51 +01:00
pierrenn 9ab14e4d21 Limit decimal range of numbers ParseScript accepts 2020-03-27 15:51:05 +09:00
Hennadii Stepanov e4d366788b
build: Drop needless EXTRA_DIST content
Some EXTRA_DIST content is needless since a git archive is used as the
source tarball.
2020-03-15 18:34:27 +02:00
MarcoFalke faf7d4fa86
build: Add cov_fuzz target 2020-02-10 05:41:34 -08:00
Wladimir J. van der Laan 402252a808 build: Add LCOV exception for crc32c 2020-01-28 17:01:48 +01:00
fanquake abc147de95
build: remove WINDOWS_BITS from build system
We no longer build/ship 32 bit windows executables.
2019-12-16 13:12:29 -05:00
nijynot f736f6920b lcov: filter /usr/lib64 from coverage report 2019-12-09 00:14:53 +01:00
nijynot a5a705b46d lcov: filter depends from coverage report 2019-12-06 19:14:15 +01:00
Wladimir J. van der Laan 4258fd7377
Merge #17091: tests: Add test for loadblock option and linearize scripts
89339d1460 tests: Add test for loadblock option (Fabian Jahr)

Pull request description:

  Fixes #17019

  Was initially part of #17044 but as the test got larger it made sense to split it into its own commit as suggested in #17019 .

  This is testing the `-loadblock` option by using the scripts in `contrib/linearize` to generate a `bootstrap.dat` file and starting a disconnected node with it. So it is also testing the linearize scripts which were untested before and needed to be made available for the CI environment, hence they are added to `DIST_CONTRIB` in `Makefile.am`.

ACKs for top commit:
  laanwj:
    ACK 89339d1460

Tree-SHA512: aede0cd6e8b21194973f3633bc07fa2672d66a6f85dfe6a57cee2bb269a65d19ea49d5f9ed7914a173b3847c76e70257aa865f44bde170c1999d9655b4862d1c
2019-10-23 11:21:46 +02:00
Fabian Jahr 89339d1460 tests: Add test for loadblock option 2019-10-13 13:09:35 +02:00
MarcoFalke fa91590a74
build: Add README.md to DIST target 2019-10-10 11:06:43 -04:00
Carl Dong 181989f6c9
build: Add variable printing target to Makefiles
I kept finding myself needing these to debug our build system, since
they are innocuous and are very helpful they probably belong in the
codebase.

Source: John Graham-Cumming
https://www.cmcrossroads.com/article/printing-value-makefile-variable
2019-10-09 11:12:55 -04:00
fanquake c84ff23c6d
build: ignore macOS make deploy artefacts & add them to clean-local 2019-08-14 08:07:12 +08:00
Aseem Sood 9218ce8d48 Failing functional tests stop lcov 2019-06-13 11:39:15 -04:00
MarcoFalke 11e32e2fd1
Merge #12051: add missing debian contrib file to tarball
5d7ce74ab3 add missing debian contrib files to tarball (Peter Wagner)

Pull request description:

  the current release is missing the debian contrib folder, add it

ACKs for commit 5d7ce7:

Tree-SHA512: 9d38c9ec0cc13171582c0bde57a2f69b22026a91f353e20da556cb63a4cfbba68b2465c9c62eaa98df50a65d971cc4411ffee519824b34068772ae8ddedb7d4c
2019-04-29 08:48:39 -04:00
Peter Wagner 5d7ce74ab3 add missing debian contrib files to tarball 2019-04-29 08:00:18 -04:00
MarcoFalke 31f7c6dd21
Merge #15295: fuzz: Add test/fuzz/test_runner.py and run it in travis
fa535af92c fuzz: test_runner: Better error message when built with afl (MarcoFalke)
fa7ca8ef58 qa: Add test/fuzz/test_runner.py (MarcoFalke)

Pull request description:

  Can be run with `./test/fuzz/test_runner.py` after building as described in `doc/fuzzing.md`

Tree-SHA512: f6a3cd8165ec2de4b363be4fd0a936b4a60829cce923f93fe5d6a046b1bbd64c959cdf790440bf70c0e13b0bb1b956a746a24c6fd92bddeab15b837ed50ffad2
2019-02-14 16:32:26 -05:00
Sjors Provoost 4a43eb8cdf
[build] Makefile.am: add rule for src/bitcoin-wallet 2019-02-14 16:17:55 +01:00
MarcoFalke fa7ca8ef58
qa: Add test/fuzz/test_runner.py 2019-02-13 17:12:28 -05:00