Commit Graph

193 Commits

Author SHA1 Message Date
J. Nick Koston 8b067e83f7
Initial orjson support take 3 (#73849)
* Initial orjson support take 2

Still need to work out problem building wheels

--

Redux of #72754 / #32153 Now possible since the following is solved:
ijl/orjson#220 (comment)

This implements orjson where we use our default encoder.  This does not implement orjson where `ExtendedJSONEncoder` is used as these areas tend to be called far less frequently.  If its desired, this could be done in a followup, but it seemed like a case of diminishing returns (except maybe for large diagnostics files, or traces, but those are not expected to be downloaded frequently).

Areas where this makes a perceptible difference:
- Anything that subscribes to entities (Initial subscribe_entities payload)
- Initial download of registries on first connection / restore
- History queries
- Saving states to the database
- Large logbook queries
- Anything that subscribes to events (appdaemon)

Cavets:
orjson supports serializing dataclasses natively (and much faster) which
eliminates the need to implement `as_dict` in many places
when the data is already in a dataclass. This works
well as long as all the data in the dataclass can also
be serialized. I audited all places where we have an `as_dict`
for a dataclass and found only backups needs to be adjusted (support for `Path` needed to be added for backups).  I was a little bit worried about `SensorExtraStoredData` with `Decimal` but it all seems to work out from since it converts it before it gets to the json encoding cc @dgomes

If it turns out to be a problem we can disable this
with option |= [orjson.OPT_PASSTHROUGH_DATACLASS](https://github.com/ijl/orjson#opt_passthrough_dataclass) and it
will fallback to `as_dict`

Its quite impressive for history queries
<img width="1271" alt="Screen_Shot_2022-05-30_at_23_46_30" src="https://user-images.githubusercontent.com/663432/171145699-661ad9db-d91d-4b2d-9c1a-9d7866c03a73.png">

* use for views as well

* handle UnicodeEncodeError

* tweak

* DRY

* DRY

* not needed

* fix tests

* Update tests/components/http/test_view.py

* Update tests/components/http/test_view.py

* black

* templates
2022-06-22 21:59:51 +02:00
Martin Hjelmare 8049170e5a
Initialize hass.config_entries for check config (#73575) 2022-06-16 14:40:41 +02:00
J. Nick Koston dca4d3cd61
Significantly improve yaml load times when the C loader is available (#73337) 2022-06-13 08:44:46 -10:00
J. Nick Koston c365454afb
Revert "Initial orjson support (#72754)" (#72789)
This was causing the wheels to fail to build. We need
to workout why when we don't have release pressure

This reverts commit d9d22a9556.
2022-05-31 13:51:55 -07:00
J. Nick Koston d9d22a9556
Initial orjson support (#72754) 2022-05-31 12:18:11 -07:00
Franck Nijhof 8f4caf4141
Clean up accessing event helpers via hass (#72011) 2022-05-17 19:36:29 +02:00
J. Nick Koston 24d7a464e1
Refactor logbook to reduce overhead and complexity (#71509) 2022-05-08 21:47:26 -07:00
J. Nick Koston b8442d9340
Add json decode caching to logbook (#71080) 2022-05-01 19:33:31 -05:00
J. Nick Koston fe6a4bfb1d
Remove EVENT_TIME_CHANGED and EVENT_TIMER_OUT_OF_SYNC (#69643)
Co-authored-by: Martin Hjelmare <marhje52@gmail.com>
2022-04-09 09:05:54 -10:00
Marc Mueller be7ef6115c
Make TypeVars private (1) (#68205) 2022-03-17 18:52:38 +01:00
Franck Nijhof fa09cf663e
Update black to 22.1.0 (#65788) 2022-02-05 14:19:37 +01:00
Tobias Sauerwein 2c904c0974
Bump mypy to 0.930 (#62642)
Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
2021-12-27 17:55:17 +01:00
Erik Montnemery cb2c2d98c3
Remove unnecessary checks before calling os.makedirs (#62576) 2021-12-23 09:59:31 +01:00
Franck Nijhof ca2f343c51
Upgrade colorlog to 6.6.0 (#59440) 2021-11-09 21:30:23 +02:00
Marc Mueller 77ee72cbb9
Import Callable from collections.abc (1) (#56775) 2021-09-29 16:32:11 +02:00
Franck Nijhof 237efcf6b1
Upgrade colorlog to 6.4.1 (#56573) 2021-09-23 16:12:13 +02:00
Paulus Schoutsen 7111fc47c4
Better handle invalid trigger config (#55637) 2021-09-03 10:15:57 -07:00
Ville Skyttä 848c0be58a
Avoid some implicit generic Anys (#54577) 2021-08-16 23:12:06 +02:00
Marc Mueller c07646db5d
Update typing syntax (#49480)
* Update typing syntax

* Replace typing imports with ones from collections where possible

* Changes after review
2021-04-20 17:40:41 +02:00
Franck Nijhof 63fa9f7dd8
Upgrade colorlog to 5.0.1 (#49221) 2021-04-14 23:56:32 +02:00
Franck Nijhof 6cead320de
Bump colorlog to 4.8.0 (#48218) 2021-03-22 13:10:01 +01:00
Marc Mueller fabd73f08b
Update typing 03 (#48015) 2021-03-17 21:46:07 +01:00
Paulus Schoutsen 2df644c6cc
Clean up secret loading (#47034) 2021-03-02 12:58:53 -08:00
Paulus Schoutsen 633a7aeb22
Remove deprecated credstash + keyring (#47033) 2021-02-25 09:48:19 +01:00
J. Nick Koston c9df42b69a
Add support for pre-filtering events to the event bus (#46371) 2021-02-14 09:42:55 -10:00
Fabian Affolter d38d8a542d
Upgrade colorlog to 4.7.2 (#45840) 2021-02-01 19:01:19 -05:00
Fabian Affolter 2e62e0661b
Upgrade colorlog to 4.6.2 (#44652) 2020-12-30 17:12:51 +01:00
Franck Nijhof 2656e9b0f9
Upgrade colorlog to 4.5.0 (#42965) 2020-11-08 14:05:55 +01:00
Ville Skyttä f28c3273c2
Upgrade mypy to 0.790 (#41595) 2020-10-12 19:17:30 -05:00
Ville Skyttä b4bac0f7a0
Exception chaining and wrapping improvements (#39320)
* Remove unnecessary exception re-wraps

* Preserve exception chains on re-raise

We slap "from cause" to almost all possible cases here. In some cases it
could conceivably be better to do "from None" if we really want to hide
the cause. However those should be in the minority, and "from cause"
should be an improvement over the corresponding raise without a "from"
in all cases anyway.

The only case where we raise from None here is in plex, where the
exception for an original invalid SSL cert is not the root cause for
failure to validate a newly fetched one.

Follow local convention on exception variable names if there is a
consistent one, otherwise `err` to match with majority of codebase.

* Fix mistaken re-wrap in homematicip_cloud/hap.py

Missed the difference between HmipConnectionError and
HmipcConnectionError.

* Do not hide original error on plex new cert validation error

Original is not the cause for the new one, but showing old in the
traceback is useful nevertheless.
2020-08-28 13:50:32 +02:00
J. Nick Koston 6b7a7939d2
Include the first seen context data in the logbook api (#39194)
* Include the context_entity_id in the logbook api

context_entity_id is the first entity seen during
a time period that includes the context

* update test

* more of them

* include friendly name

* pylint wants a ternary

* Refactor

* performance

* fix homekit context

* Fix self describing events

* Fix external_events
2020-08-24 12:44:40 -05:00
Fabian Affolter 1126c750e1
Upgrade colorlog to 4.2.1 (#39159) 2020-08-23 12:25:54 +02:00
Paulus Schoutsen f49ce5d1b4
Protect loop set default executor (#37438)
Co-authored-by: J. Nick Koston <nick@koston.org>
2020-07-06 15:58:53 -07:00
J. Nick Koston 34ccb6588c
Cleanup async_track_state_change and augment docstring (#37251)
* Cleanup async_track_state_change and augment docstrings.

Skip from_state and to_state matching in
async_track_state_change when they are None

Optimize the state change listener for the most
common use case: no to_state and from_state
matching.

* Update benchmark to be more realistic (previously we assumed only one entity was present in the whole instance)

* Add more tests to ensure behavior is preserved

* Ensure new behavior matches test

* remove MATCH_ALL from zone automation since its the default anyways

* Might as well use async_track_state_change_event instead since MATCH_ALL is removed
2020-07-05 17:31:33 -05:00
J. Nick Koston 4b2ebf5487
Ensure removed entities are not displayed in logbook (#37395) 2020-07-03 23:08:46 -07:00
mdegat01 6c7355785a
Add support for glob matching to entity filters (#36913)
* Added GLOB capability to entityfilter and every place that uses it. All existing tests are passing

* added tests for components affected by glob change

* fixed flake8 error

* mocking the correct listener

* mocking correct bus method in azure test

* tests passing in 3.7 and 3.8

* fixed formatting issue from rebase/conflict

* Checking against glob patterns in more performant way

* perf improvments and reverted unnecessarily adjusted tests

* added new benchmark test around filters

* no longer using get with default in entityfilter

* changed filter name and removed logbook from filter benchmark

* simplified benchmark tests from feedback

* fixed apache tests and returned include exclude schemas to normal

* fixed azure event hub tests to properly go through component logic

* fixed azure test and clean up for other tests

* renaming test files to match standard

* merged mqtt statestream test changes with base

* removed dependency on recorder filter schema from history

* fixed recorder tests after merge and a bunch of lint errors
2020-06-23 20:02:29 -05:00
J. Nick Koston 59e43ab6e4
Prefilter more logbook events in sql (#36958)
* Prefilter more logbook events in sql

Prefilter sensor events in _keep_event before humanify

Cache static attribute lookup

Reduces logbook execution time by ~35%

* fix mocking in benchmark

* Update tests for logbook users
2020-06-21 10:50:58 -07:00
Ville Skyttä 752679c55d
Check isinstance on collections.abc, not typing classes (#35087) 2020-05-02 23:57:48 +02:00
Fabian Affolter 1e8281fb06
Upgrade keyring to 21.2.0 (#34084) 2020-04-12 13:36:34 +02:00
springstan 46bbe816f6
Remove None from dict.get(key, None) (#33794) 2020-04-07 21:06:05 +02:00
springstan fca90a8ddc
Improve string formatting v5 (#33697)
* Improve string formatting v5

* Address review comments
2020-04-05 17:48:55 +02:00
springstan dde93304d3
Improve string formatting v2 (#33645)
* Improve string formatting v2

* Improve string formatting v3

* Address review comments
2020-04-04 23:09:34 +02:00
Franck Nijhof 22ae498f3a
Remove redundant open modes (#33652) 2020-04-04 22:49:15 +02:00
Franck Nijhof 886308a953
String formatting improvements (#33653) 2020-04-04 21:39:22 +02:00
Franck Nijhof b9b1cee403
Enable pylint import-outside-toplevel (#33631) 2020-04-04 17:07:36 +02:00
Ville Skyttä 221d5205e4
Upgrade mypy to 0.770, tighten config a bit (#32715)
* Upgrade mypy to 0.770, related cleanups

https://mypy-lang.blogspot.com/2020/03/mypy-0770-released.html

* Clean up config and make it a notch stricter, address findings
2020-03-12 11:52:20 +01:00
Paulus Schoutsen d28d1ff657
Add JSON benchmark (#32690)
* Add JSON benchmark

* Fix logbook benchmarks

* Move logbook import back
2020-03-11 11:50:16 -07:00
Paulus Schoutsen 8aea538662
Allow teaching logbook about events (#32444)
* Allow teaching logbook about events

* Use async_add_executor_job

* Fix tests
2020-03-05 11:55:50 -08:00
Paulus Schoutsen 07fa844c43
Speed up validate_entity_id (#32137)
* Speed up validate_entity_id

* Add some more invalid entity IDs

* Adjust regular expression

* Extend and sort test cases

* Update regular expression, more cases, faster

* Adjust tests, allow start with number, disallow double underscore

Co-authored-by: Franck Nijhof <frenck@frenck.nl>
2020-02-24 08:35:02 -08:00
Paulus Schoutsen 5fdc60e067
Add Safe Mode (#30723)
* Store last working HTTP settings

* Add safe mode

* Fix tests

* Add cloud to safe mode

* Update logging text

* Fix camera tests leaving files behind

* Make emulated_hue tests not leave files behind

* Make logbook tests not leave files behind

* Make tts tests not leave files behind

* Make image_processing tests not leave files behind

* Make manual_mqtt tests not leave files behind
2020-01-14 13:03:02 -08:00