heimoshuiyu/rust-synapse-compress-state
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
78 Commits 9 Branches 3 Tags
08d312b66d81498b818109434790c45bb5a8da63
Commit Graph

78 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Erik Johnston
08d312b66d Move synapse_auto_compressor 2021-09-28 16:05:38 +01:00
Erik Johnston
70b73489c0 Move auto_compressor to synapse_auto_compressor 2021-09-28 15:59:15 +01:00
Erik Johnston
4c4a93498f Add pypi metadata 2021-09-28 15:57:48 +01:00
Azrenbeth
bf57e81f54 Log to stderr not to a file (#74) 2021-09-28 14:39:18 +01:00
Azrenbeth
dcfa67ea98 Add documentation for autocompressor (#70) 2021-09-28 13:38:01 +01:00
Azrenbeth
9bfd786f3b Add pyo3 bindings to autocompressor so can be used from python (#69) 2021-09-28 09:32:50 +01:00
Azrenbeth
f822da9625 Add a binary crate to the autocompressor (#68) 2021-09-27 14:22:26 +01:00
Azrenbeth
b8e323c321 All printing replaced by logging (#67) 2021-09-27 12:20:12 +01:00
Azrenbeth
bc30942e2d Add no-progress-bars feature so other packages can hide them (#66) 2021-09-27 11:35:40 +01:00
Azrenbeth
a069d8765a Add method that compresses the chunks with lowest uncompressed state_group ids (#72) 2021-09-27 09:49:34 +01:00
Azrenbeth
3271221311 Add method that compresses next chunk of room (#64) 2021-09-20 09:21:59 +01:00
Azrenbeth
a9bc800b87 Add new package with methods to save and load compressor state (#63) 2021-09-16 09:55:14 +01:00
Azrenbeth
80795aa813 Dont send no-changes transaction to database (#71) 2021-09-15 10:09:43 +01:00
Azrenbeth
55ee83ce13 Don't panic in continue_run if no groups found within range (#62) 2021-09-14 17:29:48 +01:00
Azrenbeth
a409cdbd8e Rename level current to head (#61) 2021-09-14 12:28:35 +01:00
Azrenbeth
8c72a0de52 Move setting up jemalloc out of library crate (#60) 2021-09-14 11:26:21 +01:00
Azrenbeth
a951ba4dae Move the graph generation to before abort (#56) 2021-09-13 13:23:48 +01:00
Azrenbeth
4c3d6bd346 Add method to run the compressor starting from a particular compressor-state (#55) 2021-09-13 10:25:52 +01:00
Azrenbeth
d32f49303b Add integration tests that check various config options (#54) 2021-09-09 14:26:14 +01:00
Azrenbeth
d908d13f8f Add option to commit changes to the database automatically (#53) 2021-09-08 11:39:57 +01:00
Azrenbeth
65861de06e Stops the compressor if it will lead to more rows in the database (#52) 2021-09-06 10:06:47 +01:00
Azrenbeth
3b5d7fd176 Add information on running options to README and --help output (#51) 2021-09-06 09:59:04 +01:00
Azrenbeth
0f7f2c2660 Setup framework to do integration testing (#50) 2021-09-06 09:52:13 +01:00
Azrenbeth
011f9f8da5 Wrote tests for Stats (#49) 
wrote tests for Stats struct and whether the compressor accurately populates it
 2021-08-16 15:17:38 +01:00
Azrenbeth
9a59b1121c Wrote tests for compressor functions (#48) 
* Wrote tests for create_new_tree

* wrote tests for get_delta
 2021-08-16 15:10:36 +01:00
Azrenbeth
aa6137ce52 Wrote tests for Level struct functions (#47) 2021-08-16 15:04:06 +01:00
Azrenbeth
e5174f4181 Wrote tests for config::new() which is used by pyo3 code (#46) 2021-08-16 14:44:56 +01:00
Azrenbeth
5c248b5362 Wrote unit tests for functionality within lib.rs (#45) 2021-08-16 14:39:27 +01:00
Azrenbeth
63e1d6e3c9 Added option to only run the compressor on a range of state groups in a room (#44) 2021-08-12 16:01:12 +01:00
Azrenbeth
3290726990 Added option to output directed graphs of the group heirarchy before and after compression (#43) 
It outputs nodes and edges information before and after the
compressor has run - these can be visualised in a tool like Gephi

A good way to visualise what the compressor is actually doing!
 2021-08-04 11:56:42 +01:00
Azrenbeth
becb2937aa Wrapped code with pyo3 so that the libary can be used from python (#42) 2021-08-03 09:44:51 +01:00
Azrenbeth
a887107607 Add documentation (#41) 
Added documentation to lib.rs and database.rs files
 2021-07-30 10:06:39 +01:00
Azrenbeth
013080f712 Azren/moved to lib (#40) 2021-07-28 13:25:44 +01:00
Azrenbeth
31e37996ea Split up main (#39) 
Moved code from inside the main function into function calls
 2021-07-27 15:31:48 +01:00
Erik Johnston
f3b1dd0b8a Add github actions (#33) 2021-05-20 12:13:45 +01:00
Erik Johnston
b33c498e99 Update dependencies 2021-03-19 18:42:48 +00:00
Erik Johnston
d2ab661b1d Merge pull request #25 from heftig/update-deps 
Cargo.toml: Update dependencies
 2021-02-23 10:55:11 +00:00
Jan Alexander Steffens (heftig)
b07397b5e3 Cargo.toml: Update dependencies 2021-02-23 00:56:22 +01:00
Erik Johnston
4a56406fb3 Merge pull request #23 from ananace/postgres-ssl 
Support postgres with SSL requirements
 2021-02-22 14:05:16 +00:00
Alexander Olofsson
dc635bdd0e Clean up postgres client creation slightly 2021-02-02 19:29:34 +01:00
Alexander Olofsson
9720b19332 Always use tls connector for postgres 2021-02-02 18:30:27 +01:00
Alexander Olofsson
8e691aec1f Support postgres with SSL requirements 
This stupidly just strips any ?sslmode= arguments from the postgres URI
and enables SSL without verification regardless of which mode the user
specifies.
 2021-01-30 20:29:58 +01:00
Erik Johnston
1aac27ebb6 Merge pull request #12 from reivilibre/master 
Add information from #2 to README about being race-safe
v0.1.0 		2020-02-20 13:03:59 +00:00
Erik Johnston
be6cad0d61 Merge pull request #15 from heftig/modernize 
Modernize the code
 2020-02-20 13:02:49 +00:00
Jan Alexander Steffens (heftig)
ebbb8a01bf Avoid having to count rows 2020-02-14 15:38:22 +01:00
Jan Alexander Steffens (heftig)
76ddeab12c Simplify get_initial_data_from_db 
From Jörg Sommer's PR:
https://github.com/matrix-org/rust-synapse-compress-state/pull/10
 2020-02-14 15:38:22 +01:00
Jörg Sommer
c31ba2750f main: new argument -m to give a minimal saving 
Some compressions yield only a few hundred lines saving which isn't worth
the database update and the time needed to validate the compression. With
the new commandline argument `-m` the output would be suppressed, if not
at least the given number of database records would be saved.
 2020-02-14 15:38:21 +01:00
Jörg Sommer
88fc47d819 main: add value_name() for max_state_group 
To unify the spelling of parameters, i.e. all upper-casse, add the
`value_name()` for max_state_group.
 2020-02-14 15:38:21 +01:00
Jörg Sommer
85d185fa98 Use character for split() 
`cargo clippy` suggested to use a character instead of a string with a
single character.

See: <https://rust-lang.github.io/rust-clippy/master/index.html#single_char_pattern>
 2020-02-14 15:38:21 +01:00
Jörg Sommer
846ec8f898 Convert len() > 0 to ! is_empty() 
`cargo clippy` suggested to use `is_empty()` instead of `len() > 0`,
because this doesn't require to calculate the whole length of the sequence.

See: <https://rust-lang.github.io/rust-clippy/master/index.html#len_zero>
 2020-02-14 15:38:21 +01:00