heimoshuiyu/rust-synapse-compress-state
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
81 Commits 9 Branches 3 Tags
38d800a775efc3645ab89ee55cb5e63922365e63
Commit Graph
81 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Sean QuahandGitHub 38d800a775 Configure @matrix-org/synapse-core to be the code owner for the repo (#76) 
Signed-off-by: Sean Quah <seanq@element.io>
 2021-10-25 14:32:07 +01:00
Erik Johnston 5272acedd2 Merge pull request #32 2021-10-13 10:19:25 +01:00
Erik Johnston 9d642cfb67 Release v0.1.2 of auto compressor v0.1.2 2021-10-06 10:55:47 +01:00
Erik JohnstonandGitHub 0111079153 Make the auto compressor uploadable to pypi (#75) 2021-09-28 16:57:13 +01:00
AzrenbethandGitHub bf57e81f54 Log to stderr not to a file (#74) 2021-09-28 14:39:18 +01:00
AzrenbethandGitHub dcfa67ea98 Add documentation for autocompressor (#70) 2021-09-28 13:38:01 +01:00
AzrenbethandGitHub 9bfd786f3b Add pyo3 bindings to autocompressor so can be used from python (#69) 2021-09-28 09:32:50 +01:00
AzrenbethandGitHub f822da9625 Add a binary crate to the autocompressor (#68) 2021-09-27 14:22:26 +01:00
AzrenbethandGitHub b8e323c321 All printing replaced by logging (#67) 2021-09-27 12:20:12 +01:00
AzrenbethandGitHub bc30942e2d Add no-progress-bars feature so other packages can hide them (#66) 2021-09-27 11:35:40 +01:00
AzrenbethandGitHub a069d8765a Add method that compresses the chunks with lowest uncompressed state_group ids (#72) 2021-09-27 09:49:34 +01:00
AzrenbethandGitHub 3271221311 Add method that compresses next chunk of room (#64) 2021-09-20 09:21:59 +01:00
AzrenbethandGitHub a9bc800b87 Add new package with methods to save and load compressor state (#63) 2021-09-16 09:55:14 +01:00
AzrenbethandGitHub 80795aa813 Dont send no-changes transaction to database (#71) 2021-09-15 10:09:43 +01:00
AzrenbethandGitHub 55ee83ce13 Don't panic in continue_run if no groups found within range (#62) 2021-09-14 17:29:48 +01:00
AzrenbethandGitHub a409cdbd8e Rename level current to head (#61) 2021-09-14 12:28:35 +01:00
AzrenbethandGitHub 8c72a0de52 Move setting up jemalloc out of library crate (#60) 2021-09-14 11:26:21 +01:00
AzrenbethandGitHub a951ba4dae Move the graph generation to before abort (#56) 2021-09-13 13:23:48 +01:00
AzrenbethandGitHub 4c3d6bd346 Add method to run the compressor starting from a particular compressor-state (#55) 2021-09-13 10:25:52 +01:00
AzrenbethandGitHub d32f49303b Add integration tests that check various config options (#54) 2021-09-09 14:26:14 +01:00
AzrenbethandGitHub d908d13f8f Add option to commit changes to the database automatically (#53) 2021-09-08 11:39:57 +01:00
AzrenbethandGitHub 65861de06e Stops the compressor if it will lead to more rows in the database (#52) 2021-09-06 10:06:47 +01:00
AzrenbethandGitHub 3b5d7fd176 Add information on running options to README and --help output (#51) 2021-09-06 09:59:04 +01:00
AzrenbethandGitHub 0f7f2c2660 Setup framework to do integration testing (#50) 2021-09-06 09:52:13 +01:00
AzrenbethandGitHub 011f9f8da5 Wrote tests for Stats (#49) 
wrote tests for Stats struct and whether the compressor accurately populates it
 2021-08-16 15:17:38 +01:00
AzrenbethandGitHub 9a59b1121c Wrote tests for compressor functions (#48) 
* Wrote tests for create_new_tree

* wrote tests for get_delta
 2021-08-16 15:10:36 +01:00
AzrenbethandGitHub aa6137ce52 Wrote tests for Level struct functions (#47) 2021-08-16 15:04:06 +01:00
AzrenbethandGitHub e5174f4181 Wrote tests for config::new() which is used by pyo3 code (#46) 2021-08-16 14:44:56 +01:00
AzrenbethandGitHub 5c248b5362 Wrote unit tests for functionality within lib.rs (#45) 2021-08-16 14:39:27 +01:00
AzrenbethandGitHub 63e1d6e3c9 Added option to only run the compressor on a range of state groups in a room (#44) 2021-08-12 16:01:12 +01:00
AzrenbethandGitHub 3290726990 Added option to output directed graphs of the group heirarchy before and after compression (#43) 
It outputs nodes and edges information before and after the
compressor has run - these can be visualised in a tool like Gephi

A good way to visualise what the compressor is actually doing!
 2021-08-04 11:56:42 +01:00
AzrenbethandGitHub becb2937aa Wrapped code with pyo3 so that the libary can be used from python (#42) 2021-08-03 09:44:51 +01:00
AzrenbethandGitHub a887107607 Add documentation (#41) 
Added documentation to lib.rs and database.rs files
 2021-07-30 10:06:39 +01:00
AzrenbethandGitHub 013080f712 Azren/moved to lib (#40) 2021-07-28 13:25:44 +01:00
AzrenbethandGitHub 31e37996ea Split up main (#39) 
Moved code from inside the main function into function calls
 2021-07-27 15:31:48 +01:00
Erik JohnstonandGitHub f3b1dd0b8a Add github actions (#33) 2021-05-20 12:13:45 +01:00
Jörg Sommer 2fc2db2848 Update dependencies 2021-05-13 17:25:48 +02:00
Jörg Sommer 4f823fba78 Use saturating arithmetic on calculating row saving 
The saturating arithmetic doesn't overflow, but stays at the maximum or
minimum. In this case it doesn't become negative, but zero, which
satisfies the condition below.

Closes #31
 2021-05-13 17:25:37 +02:00
Erik Johnston b33c498e99 Update dependencies 2021-03-19 18:42:48 +00:00
Erik JohnstonandGitHub d2ab661b1d Merge pull request #25 from heftig/update-deps 
Cargo.toml: Update dependencies
 2021-02-23 10:55:11 +00:00
Jan Alexander Steffens (heftig) b07397b5e3 Cargo.toml: Update dependencies 2021-02-23 00:56:22 +01:00
Erik JohnstonandGitHub 4a56406fb3 Merge pull request #23 from ananace/postgres-ssl 
Support postgres with SSL requirements
 2021-02-22 14:05:16 +00:00
Alexander Olofsson dc635bdd0e Clean up postgres client creation slightly 2021-02-02 19:29:34 +01:00
Alexander Olofsson 9720b19332 Always use tls connector for postgres 2021-02-02 18:30:27 +01:00
Alexander Olofsson 8e691aec1f Support postgres with SSL requirements 
This stupidly just strips any ?sslmode= arguments from the postgres URI
and enables SSL without verification regardless of which mode the user
specifies.
 2021-01-30 20:29:58 +01:00
Erik JohnstonandGitHub 1aac27ebb6 Merge pull request #12 from reivilibre/master 
Add information from #2 to README about being race-safe
v0.1.0 		2020-02-20 13:03:59 +00:00
Erik JohnstonandGitHub be6cad0d61 Merge pull request #15 from heftig/modernize 
Modernize the code
 2020-02-20 13:02:49 +00:00
Jan Alexander Steffens (heftig) ebbb8a01bf Avoid having to count rows 2020-02-14 15:38:22 +01:00
Jan Alexander Steffens (heftig) 76ddeab12c Simplify get_initial_data_from_db 
From Jörg Sommer's PR:
https://github.com/matrix-org/rust-synapse-compress-state/pull/10
 2020-02-14 15:38:22 +01:00
Jörg SommerandJan Alexander Steffens (heftig) c31ba2750f main: new argument -m to give a minimal saving 
Some compressions yield only a few hundred lines saving which isn't worth
the database update and the time needed to validate the compression. With
the new commandline argument `-m` the output would be suppressed, if not
at least the given number of database records would be saved.
 2020-02-14 15:38:21 +01:00