heimoshuiyu/rust-synapse-compress-state
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
110 Commits 9 Branches 3 Tags
982ee5ead839938c90fa5d0ac871ae31393ba891
Commit Graph

66 Commits

Author SHA1 Message Date
Jan Alexander Steffens
9ee99cd547 Fix clippy warnings, update dependencies (4) (#118) 
Signed-off-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org>
 2023-06-06 16:45:49 +01:00
msrd0
6a065de6fc Make jemalloc, clap and pyo3 optional dependencies (#116) 2023-04-21 13:17:35 +01:00
Erik Johnston
f4d96c73a8 Fix clippy linting (#115) 
* Fix clippy

(This was fixed via `cargo clippy --fix`)

* Fmt
 2023-03-27 12:00:32 +01:00
Jan Alexander Steffens
152808baca Fix clippy warnings, update dependencies (3) (#106) 2022-10-17 13:43:39 +01:00
Jan Alexander Steffens
da6271a331 Fix clippy warnings, update dependencies (again) (#100) 2022-08-03 10:52:47 +01:00
Jan Alexander Steffens
b4f3d8adbd Fix clippy warnings, update dependencies (#91) 2022-06-06 10:34:07 +01:00
Jan Alexander Steffens
4c7316311b Update dependencies, use tikv-jemallocator (#73) 
Signed-off-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org>
 2021-10-26 08:54:56 +01:00
Jan Alexander Steffens
83e8dedfa9 lib: New argument -N to suppress verification (#26) 
Signed-off-by: Jan Alexander Steffens (heftig) <heftig@archlinux.org>
 2021-10-26 08:50:34 +01:00
Erik Johnston
5272acedd2 Merge pull request #32 2021-10-13 10:19:25 +01:00
Erik Johnston
0111079153 Make the auto compressor uploadable to pypi (#75) 2021-09-28 16:57:13 +01:00
Azrenbeth
dcfa67ea98 Add documentation for autocompressor (#70) 2021-09-28 13:38:01 +01:00
Azrenbeth
b8e323c321 All printing replaced by logging (#67) 2021-09-27 12:20:12 +01:00
Azrenbeth
bc30942e2d Add no-progress-bars feature so other packages can hide them (#66) 2021-09-27 11:35:40 +01:00
Azrenbeth
a069d8765a Add method that compresses the chunks with lowest uncompressed state_group ids (#72) 2021-09-27 09:49:34 +01:00
Azrenbeth
80795aa813 Dont send no-changes transaction to database (#71) 2021-09-15 10:09:43 +01:00
Azrenbeth
55ee83ce13 Don't panic in continue_run if no groups found within range (#62) 2021-09-14 17:29:48 +01:00
Azrenbeth
a409cdbd8e Rename level current to head (#61) 2021-09-14 12:28:35 +01:00
Azrenbeth
8c72a0de52 Move setting up jemalloc out of library crate (#60) 2021-09-14 11:26:21 +01:00
Azrenbeth
a951ba4dae Move the graph generation to before abort (#56) 2021-09-13 13:23:48 +01:00
Azrenbeth
4c3d6bd346 Add method to run the compressor starting from a particular compressor-state (#55) 2021-09-13 10:25:52 +01:00
Azrenbeth
d32f49303b Add integration tests that check various config options (#54) 2021-09-09 14:26:14 +01:00
Azrenbeth
d908d13f8f Add option to commit changes to the database automatically (#53) 2021-09-08 11:39:57 +01:00
Azrenbeth
65861de06e Stops the compressor if it will lead to more rows in the database (#52) 2021-09-06 10:06:47 +01:00
Azrenbeth
3b5d7fd176 Add information on running options to README and --help output (#51) 2021-09-06 09:59:04 +01:00
Azrenbeth
0f7f2c2660 Setup framework to do integration testing (#50) 2021-09-06 09:52:13 +01:00
Azrenbeth
011f9f8da5 Wrote tests for Stats (#49) 
wrote tests for Stats struct and whether the compressor accurately populates it
 2021-08-16 15:17:38 +01:00
Azrenbeth
9a59b1121c Wrote tests for compressor functions (#48) 
* Wrote tests for create_new_tree

* wrote tests for get_delta
 2021-08-16 15:10:36 +01:00
Azrenbeth
aa6137ce52 Wrote tests for Level struct functions (#47) 2021-08-16 15:04:06 +01:00
Azrenbeth
e5174f4181 Wrote tests for config::new() which is used by pyo3 code (#46) 2021-08-16 14:44:56 +01:00
Azrenbeth
5c248b5362 Wrote unit tests for functionality within lib.rs (#45) 2021-08-16 14:39:27 +01:00
Azrenbeth
63e1d6e3c9 Added option to only run the compressor on a range of state groups in a room (#44) 2021-08-12 16:01:12 +01:00
Azrenbeth
3290726990 Added option to output directed graphs of the group heirarchy before and after compression (#43) 
It outputs nodes and edges information before and after the
compressor has run - these can be visualised in a tool like Gephi

A good way to visualise what the compressor is actually doing!
 2021-08-04 11:56:42 +01:00
Azrenbeth
becb2937aa Wrapped code with pyo3 so that the libary can be used from python (#42) 2021-08-03 09:44:51 +01:00
Azrenbeth
a887107607 Add documentation (#41) 
Added documentation to lib.rs and database.rs files
 2021-07-30 10:06:39 +01:00
Azrenbeth
013080f712 Azren/moved to lib (#40) 2021-07-28 13:25:44 +01:00
Azrenbeth
31e37996ea Split up main (#39) 
Moved code from inside the main function into function calls
 2021-07-27 15:31:48 +01:00
Jörg Sommer
4f823fba78 Use saturating arithmetic on calculating row saving 
The saturating arithmetic doesn't overflow, but stays at the maximum or
minimum. In this case it doesn't become negative, but zero, which
satisfies the condition below.

Closes #31
 2021-05-13 17:25:37 +02:00
Jan Alexander Steffens (heftig)
b07397b5e3 Cargo.toml: Update dependencies 2021-02-23 00:56:22 +01:00
Alexander Olofsson
dc635bdd0e Clean up postgres client creation slightly 2021-02-02 19:29:34 +01:00
Alexander Olofsson
9720b19332 Always use tls connector for postgres 2021-02-02 18:30:27 +01:00
Alexander Olofsson
8e691aec1f Support postgres with SSL requirements 
This stupidly just strips any ?sslmode= arguments from the postgres URI
and enables SSL without verification regardless of which mode the user
specifies.
 2021-01-30 20:29:58 +01:00
Jan Alexander Steffens (heftig)
ebbb8a01bf Avoid having to count rows 2020-02-14 15:38:22 +01:00
Jan Alexander Steffens (heftig)
76ddeab12c Simplify get_initial_data_from_db 
From Jörg Sommer's PR:
https://github.com/matrix-org/rust-synapse-compress-state/pull/10
 2020-02-14 15:38:22 +01:00
Jörg Sommer
c31ba2750f main: new argument -m to give a minimal saving 
Some compressions yield only a few hundred lines saving which isn't worth
the database update and the time needed to validate the compression. With
the new commandline argument `-m` the output would be suppressed, if not
at least the given number of database records would be saved.
 2020-02-14 15:38:21 +01:00
Jörg Sommer
88fc47d819 main: add value_name() for max_state_group 
To unify the spelling of parameters, i.e. all upper-casse, add the
`value_name()` for max_state_group.
 2020-02-14 15:38:21 +01:00
Jörg Sommer
85d185fa98 Use character for split() 
`cargo clippy` suggested to use a character instead of a string with a
single character.

See: <https://rust-lang.github.io/rust-clippy/master/index.html#single_char_pattern>
 2020-02-14 15:38:21 +01:00
Jörg Sommer
846ec8f898 Convert len() > 0 to ! is_empty() 
`cargo clippy` suggested to use `is_empty()` instead of `len() > 0`,
because this doesn't require to calculate the whole length of the sequence.

See: <https://rust-lang.github.io/rust-clippy/master/index.html#len_zero>
 2020-02-14 15:38:21 +01:00
Jan Alexander Steffens (heftig)
f9d3570363 Fix typo: Rename PGEscapse to PGEscape 2020-02-14 15:38:21 +01:00
Jan Alexander Steffens (heftig)
9990141d40 Ignore a warning from inside the clap macros 2020-02-14 15:38:21 +01:00
Jan Alexander Steffens (heftig)
39750c763e Update deps 2020-02-14 15:36:31 +01:00