automatically ignore known forwarded addresses, fixes #64

This commit is contained in:
Aine
2023-09-18 12:35:37 +03:00
parent e90925eceb
commit 60b4386dd8
187 changed files with 4070 additions and 2667 deletions

373
vendor/go.mau.fi/util/LICENSE vendored Normal file
View File

@@ -0,0 +1,373 @@
Mozilla Public License Version 2.0
==================================
1. Definitions
--------------
1.1. "Contributor"
means each individual or legal entity that creates, contributes to
the creation of, or owns Covered Software.
1.2. "Contributor Version"
means the combination of the Contributions of others (if any) used
by a Contributor and that particular Contributor's Contribution.
1.3. "Contribution"
means Covered Software of a particular Contributor.
1.4. "Covered Software"
means Source Code Form to which the initial Contributor has attached
the notice in Exhibit A, the Executable Form of such Source Code
Form, and Modifications of such Source Code Form, in each case
including portions thereof.
1.5. "Incompatible With Secondary Licenses"
means
(a) that the initial Contributor has attached the notice described
in Exhibit B to the Covered Software; or
(b) that the Covered Software was made available under the terms of
version 1.1 or earlier of the License, but not also under the
terms of a Secondary License.
1.6. "Executable Form"
means any form of the work other than Source Code Form.
1.7. "Larger Work"
means a work that combines Covered Software with other material, in
a separate file or files, that is not Covered Software.
1.8. "License"
means this document.
1.9. "Licensable"
means having the right to grant, to the maximum extent possible,
whether at the time of the initial grant or subsequently, any and
all of the rights conveyed by this License.
1.10. "Modifications"
means any of the following:
(a) any file in Source Code Form that results from an addition to,
deletion from, or modification of the contents of Covered
Software; or
(b) any new file in Source Code Form that contains any Covered
Software.
1.11. "Patent Claims" of a Contributor
means any patent claim(s), including without limitation, method,
process, and apparatus claims, in any patent Licensable by such
Contributor that would be infringed, but for the grant of the
License, by the making, using, selling, offering for sale, having
made, import, or transfer of either its Contributions or its
Contributor Version.
1.12. "Secondary License"
means either the GNU General Public License, Version 2.0, the GNU
Lesser General Public License, Version 2.1, the GNU Affero General
Public License, Version 3.0, or any later versions of those
licenses.
1.13. "Source Code Form"
means the form of the work preferred for making modifications.
1.14. "You" (or "Your")
means an individual or a legal entity exercising rights under this
License. For legal entities, "You" includes any entity that
controls, is controlled by, or is under common control with You. For
purposes of this definition, "control" means (a) the power, direct
or indirect, to cause the direction or management of such entity,
whether by contract or otherwise, or (b) ownership of more than
fifty percent (50%) of the outstanding shares or beneficial
ownership of such entity.
2. License Grants and Conditions
--------------------------------
2.1. Grants
Each Contributor hereby grants You a world-wide, royalty-free,
non-exclusive license:
(a) under intellectual property rights (other than patent or trademark)
Licensable by such Contributor to use, reproduce, make available,
modify, display, perform, distribute, and otherwise exploit its
Contributions, either on an unmodified basis, with Modifications, or
as part of a Larger Work; and
(b) under Patent Claims of such Contributor to make, use, sell, offer
for sale, have made, import, and otherwise transfer either its
Contributions or its Contributor Version.
2.2. Effective Date
The licenses granted in Section 2.1 with respect to any Contribution
become effective for each Contribution on the date the Contributor first
distributes such Contribution.
2.3. Limitations on Grant Scope
The licenses granted in this Section 2 are the only rights granted under
this License. No additional rights or licenses will be implied from the
distribution or licensing of Covered Software under this License.
Notwithstanding Section 2.1(b) above, no patent license is granted by a
Contributor:
(a) for any code that a Contributor has removed from Covered Software;
or
(b) for infringements caused by: (i) Your and any other third party's
modifications of Covered Software, or (ii) the combination of its
Contributions with other software (except as part of its Contributor
Version); or
(c) under Patent Claims infringed by Covered Software in the absence of
its Contributions.
This License does not grant any rights in the trademarks, service marks,
or logos of any Contributor (except as may be necessary to comply with
the notice requirements in Section 3.4).
2.4. Subsequent Licenses
No Contributor makes additional grants as a result of Your choice to
distribute the Covered Software under a subsequent version of this
License (see Section 10.2) or under the terms of a Secondary License (if
permitted under the terms of Section 3.3).
2.5. Representation
Each Contributor represents that the Contributor believes its
Contributions are its original creation(s) or it has sufficient rights
to grant the rights to its Contributions conveyed by this License.
2.6. Fair Use
This License is not intended to limit any rights You have under
applicable copyright doctrines of fair use, fair dealing, or other
equivalents.
2.7. Conditions
Sections 3.1, 3.2, 3.3, and 3.4 are conditions of the licenses granted
in Section 2.1.
3. Responsibilities
-------------------
3.1. Distribution of Source Form
All distribution of Covered Software in Source Code Form, including any
Modifications that You create or to which You contribute, must be under
the terms of this License. You must inform recipients that the Source
Code Form of the Covered Software is governed by the terms of this
License, and how they can obtain a copy of this License. You may not
attempt to alter or restrict the recipients' rights in the Source Code
Form.
3.2. Distribution of Executable Form
If You distribute Covered Software in Executable Form then:
(a) such Covered Software must also be made available in Source Code
Form, as described in Section 3.1, and You must inform recipients of
the Executable Form how they can obtain a copy of such Source Code
Form by reasonable means in a timely manner, at a charge no more
than the cost of distribution to the recipient; and
(b) You may distribute such Executable Form under the terms of this
License, or sublicense it under different terms, provided that the
license for the Executable Form does not attempt to limit or alter
the recipients' rights in the Source Code Form under this License.
3.3. Distribution of a Larger Work
You may create and distribute a Larger Work under terms of Your choice,
provided that You also comply with the requirements of this License for
the Covered Software. If the Larger Work is a combination of Covered
Software with a work governed by one or more Secondary Licenses, and the
Covered Software is not Incompatible With Secondary Licenses, this
License permits You to additionally distribute such Covered Software
under the terms of such Secondary License(s), so that the recipient of
the Larger Work may, at their option, further distribute the Covered
Software under the terms of either this License or such Secondary
License(s).
3.4. Notices
You may not remove or alter the substance of any license notices
(including copyright notices, patent notices, disclaimers of warranty,
or limitations of liability) contained within the Source Code Form of
the Covered Software, except that You may alter any license notices to
the extent required to remedy known factual inaccuracies.
3.5. Application of Additional Terms
You may choose to offer, and to charge a fee for, warranty, support,
indemnity or liability obligations to one or more recipients of Covered
Software. However, You may do so only on Your own behalf, and not on
behalf of any Contributor. You must make it absolutely clear that any
such warranty, support, indemnity, or liability obligation is offered by
You alone, and You hereby agree to indemnify every Contributor for any
liability incurred by such Contributor as a result of warranty, support,
indemnity or liability terms You offer. You may include additional
disclaimers of warranty and limitations of liability specific to any
jurisdiction.
4. Inability to Comply Due to Statute or Regulation
---------------------------------------------------
If it is impossible for You to comply with any of the terms of this
License with respect to some or all of the Covered Software due to
statute, judicial order, or regulation then You must: (a) comply with
the terms of this License to the maximum extent possible; and (b)
describe the limitations and the code they affect. Such description must
be placed in a text file included with all distributions of the Covered
Software under this License. Except to the extent prohibited by statute
or regulation, such description must be sufficiently detailed for a
recipient of ordinary skill to be able to understand it.
5. Termination
--------------
5.1. The rights granted under this License will terminate automatically
if You fail to comply with any of its terms. However, if You become
compliant, then the rights granted under this License from a particular
Contributor are reinstated (a) provisionally, unless and until such
Contributor explicitly and finally terminates Your grants, and (b) on an
ongoing basis, if such Contributor fails to notify You of the
non-compliance by some reasonable means prior to 60 days after You have
come back into compliance. Moreover, Your grants from a particular
Contributor are reinstated on an ongoing basis if such Contributor
notifies You of the non-compliance by some reasonable means, this is the
first time You have received notice of non-compliance with this License
from such Contributor, and You become compliant prior to 30 days after
Your receipt of the notice.
5.2. If You initiate litigation against any entity by asserting a patent
infringement claim (excluding declaratory judgment actions,
counter-claims, and cross-claims) alleging that a Contributor Version
directly or indirectly infringes any patent, then the rights granted to
You by any and all Contributors for the Covered Software under Section
2.1 of this License shall terminate.
5.3. In the event of termination under Sections 5.1 or 5.2 above, all
end user license agreements (excluding distributors and resellers) which
have been validly granted by You or Your distributors under this License
prior to termination shall survive termination.
************************************************************************
* *
* 6. Disclaimer of Warranty *
* ------------------------- *
* *
* Covered Software is provided under this License on an "as is" *
* basis, without warranty of any kind, either expressed, implied, or *
* statutory, including, without limitation, warranties that the *
* Covered Software is free of defects, merchantable, fit for a *
* particular purpose or non-infringing. The entire risk as to the *
* quality and performance of the Covered Software is with You. *
* Should any Covered Software prove defective in any respect, You *
* (not any Contributor) assume the cost of any necessary servicing, *
* repair, or correction. This disclaimer of warranty constitutes an *
* essential part of this License. No use of any Covered Software is *
* authorized under this License except under this disclaimer. *
* *
************************************************************************
************************************************************************
* *
* 7. Limitation of Liability *
* -------------------------- *
* *
* Under no circumstances and under no legal theory, whether tort *
* (including negligence), contract, or otherwise, shall any *
* Contributor, or anyone who distributes Covered Software as *
* permitted above, be liable to You for any direct, indirect, *
* special, incidental, or consequential damages of any character *
* including, without limitation, damages for lost profits, loss of *
* goodwill, work stoppage, computer failure or malfunction, or any *
* and all other commercial damages or losses, even if such party *
* shall have been informed of the possibility of such damages. This *
* limitation of liability shall not apply to liability for death or *
* personal injury resulting from such party's negligence to the *
* extent applicable law prohibits such limitation. Some *
* jurisdictions do not allow the exclusion or limitation of *
* incidental or consequential damages, so this exclusion and *
* limitation may not apply to You. *
* *
************************************************************************
8. Litigation
-------------
Any litigation relating to this License may be brought only in the
courts of a jurisdiction where the defendant maintains its principal
place of business and such litigation shall be governed by laws of that
jurisdiction, without reference to its conflict-of-law provisions.
Nothing in this Section shall prevent a party's ability to bring
cross-claims or counter-claims.
9. Miscellaneous
----------------
This License represents the complete agreement concerning the subject
matter hereof. If any provision of this License is held to be
unenforceable, such provision shall be reformed only to the extent
necessary to make it enforceable. Any law or regulation which provides
that the language of a contract shall be construed against the drafter
shall not be used to construe this License against a Contributor.
10. Versions of the License
---------------------------
10.1. New Versions
Mozilla Foundation is the license steward. Except as provided in Section
10.3, no one other than the license steward has the right to modify or
publish new versions of this License. Each version will be given a
distinguishing version number.
10.2. Effect of New Versions
You may distribute the Covered Software under the terms of the version
of the License under which You originally received the Covered Software,
or under the terms of any subsequent version published by the license
steward.
10.3. Modified Versions
If you create software not governed by this License, and you want to
create a new license for such software, you may create and use a
modified version of this License if you rename the license and remove
any references to the name of the license steward (except to note that
such modified license differs from this License).
10.4. Distributing Source Code Form that is Incompatible With Secondary
Licenses
If You choose to distribute Source Code Form that is Incompatible With
Secondary Licenses under the terms of this version of the License, the
notice described in Exhibit B of this License must be attached.
Exhibit A - Source Code Form License Notice
-------------------------------------------
This Source Code Form is subject to the terms of the Mozilla Public
License, v. 2.0. If a copy of the MPL was not distributed with this
file, You can obtain one at http://mozilla.org/MPL/2.0/.
If it is not possible or desirable to put the notice in a particular
file, then You may include the notice in a location (such as a LICENSE
file in a relevant directory) where a recipient would be likely to look
for such a notice.
You may add additional accurate notices of copyright ownership.
Exhibit B - "Incompatible With Secondary Licenses" Notice
---------------------------------------------------------
This Source Code Form is "Incompatible With Secondary Licenses", as
defined by the Mozilla Public License, v. 2.0.

View File

@@ -0,0 +1,9 @@
base58
==========
This is a copy of <https://github.com/btcsuite/btcd/tree/master/btcutil/base58>.
## License
Package base58 is licensed under the [copyfree](http://copyfree.org) ISC
License.

View File

@@ -0,0 +1,49 @@
// Copyright (c) 2015 The btcsuite developers
// Use of this source code is governed by an ISC
// license that can be found in the LICENSE file.
// AUTOGENERATED by genalphabet.go; do not edit.
package base58
const (
// alphabet is the modified base58 alphabet used by Bitcoin.
alphabet = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz"
alphabetIdx0 = '1'
)
var b58 = [256]byte{
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 0, 1, 2, 3, 4, 5, 6,
7, 8, 255, 255, 255, 255, 255, 255,
255, 9, 10, 11, 12, 13, 14, 15,
16, 255, 17, 18, 19, 20, 21, 255,
22, 23, 24, 25, 26, 27, 28, 29,
30, 31, 32, 255, 255, 255, 255, 255,
255, 33, 34, 35, 36, 37, 38, 39,
40, 41, 42, 43, 255, 44, 45, 46,
47, 48, 49, 50, 51, 52, 53, 54,
55, 56, 57, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
255, 255, 255, 255, 255, 255, 255, 255,
}

138
vendor/go.mau.fi/util/base58/base58.go vendored Normal file
View File

@@ -0,0 +1,138 @@
// Copyright (c) 2013-2015 The btcsuite developers
// Use of this source code is governed by an ISC
// license that can be found in the LICENSE file.
package base58
import (
"math/big"
)
//go:generate go run genalphabet.go
var bigRadix = [...]*big.Int{
big.NewInt(0),
big.NewInt(58),
big.NewInt(58 * 58),
big.NewInt(58 * 58 * 58),
big.NewInt(58 * 58 * 58 * 58),
big.NewInt(58 * 58 * 58 * 58 * 58),
big.NewInt(58 * 58 * 58 * 58 * 58 * 58),
big.NewInt(58 * 58 * 58 * 58 * 58 * 58 * 58),
big.NewInt(58 * 58 * 58 * 58 * 58 * 58 * 58 * 58),
big.NewInt(58 * 58 * 58 * 58 * 58 * 58 * 58 * 58 * 58),
bigRadix10,
}
var bigRadix10 = big.NewInt(58 * 58 * 58 * 58 * 58 * 58 * 58 * 58 * 58 * 58) // 58^10
// Decode decodes a modified base58 string to a byte slice.
func Decode(b string) []byte {
answer := big.NewInt(0)
scratch := new(big.Int)
// Calculating with big.Int is slow for each iteration.
// x += b58[b[i]] * j
// j *= 58
//
// Instead we can try to do as much calculations on int64.
// We can represent a 10 digit base58 number using an int64.
//
// Hence we'll try to convert 10, base58 digits at a time.
// The rough idea is to calculate `t`, such that:
//
// t := b58[b[i+9]] * 58^9 ... + b58[b[i+1]] * 58^1 + b58[b[i]] * 58^0
// x *= 58^10
// x += t
//
// Of course, in addition, we'll need to handle boundary condition when `b` is not multiple of 58^10.
// In that case we'll use the bigRadix[n] lookup for the appropriate power.
for t := b; len(t) > 0; {
n := len(t)
if n > 10 {
n = 10
}
total := uint64(0)
for _, v := range t[:n] {
tmp := b58[v]
if tmp == 255 {
return []byte("")
}
total = total*58 + uint64(tmp)
}
answer.Mul(answer, bigRadix[n])
scratch.SetUint64(total)
answer.Add(answer, scratch)
t = t[n:]
}
tmpval := answer.Bytes()
var numZeros int
for numZeros = 0; numZeros < len(b); numZeros++ {
if b[numZeros] != alphabetIdx0 {
break
}
}
flen := numZeros + len(tmpval)
val := make([]byte, flen)
copy(val[numZeros:], tmpval)
return val
}
// Encode encodes a byte slice to a modified base58 string.
func Encode(b []byte) string {
x := new(big.Int)
x.SetBytes(b)
// maximum length of output is log58(2^(8*len(b))) == len(b) * 8 / log(58)
maxlen := int(float64(len(b))*1.365658237309761) + 1
answer := make([]byte, 0, maxlen)
mod := new(big.Int)
for x.Sign() > 0 {
// Calculating with big.Int is slow for each iteration.
// x, mod = x / 58, x % 58
//
// Instead we can try to do as much calculations on int64.
// x, mod = x / 58^10, x % 58^10
//
// Which will give us mod, which is 10 digit base58 number.
// We'll loop that 10 times to convert to the answer.
x.DivMod(x, bigRadix10, mod)
if x.Sign() == 0 {
// When x = 0, we need to ensure we don't add any extra zeros.
m := mod.Int64()
for m > 0 {
answer = append(answer, alphabet[m%58])
m /= 58
}
} else {
m := mod.Int64()
for i := 0; i < 10; i++ {
answer = append(answer, alphabet[m%58])
m /= 58
}
}
}
// leading zero bytes
for _, i := range b {
if i != 0 {
break
}
answer = append(answer, alphabetIdx0)
}
// reverse
alen := len(answer)
for i := 0; i < alen/2; i++ {
answer[i], answer[alen-1-i] = answer[alen-1-i], answer[i]
}
return string(answer)
}

View File

@@ -0,0 +1,52 @@
// Copyright (c) 2013-2014 The btcsuite developers
// Use of this source code is governed by an ISC
// license that can be found in the LICENSE file.
package base58
import (
"crypto/sha256"
"errors"
)
// ErrChecksum indicates that the checksum of a check-encoded string does not verify against
// the checksum.
var ErrChecksum = errors.New("checksum error")
// ErrInvalidFormat indicates that the check-encoded string has an invalid format.
var ErrInvalidFormat = errors.New("invalid format: version and/or checksum bytes missing")
// checksum: first four bytes of sha256^2
func checksum(input []byte) (cksum [4]byte) {
h := sha256.Sum256(input)
h2 := sha256.Sum256(h[:])
copy(cksum[:], h2[:4])
return
}
// CheckEncode prepends a version byte and appends a four byte checksum.
func CheckEncode(input []byte, version byte) string {
b := make([]byte, 0, 1+len(input)+4)
b = append(b, version)
b = append(b, input...)
cksum := checksum(b)
b = append(b, cksum[:]...)
return Encode(b)
}
// CheckDecode decodes a string that was encoded with CheckEncode and verifies the checksum.
func CheckDecode(input string) (result []byte, version byte, err error) {
decoded := Decode(input)
if len(decoded) < 5 {
return nil, 0, ErrInvalidFormat
}
version = decoded[0]
var cksum [4]byte
copy(cksum[:], decoded[len(decoded)-4:])
if checksum(decoded[:len(decoded)-4]) != cksum {
return nil, 0, ErrChecksum
}
payload := decoded[1 : len(decoded)-4]
result = append(result, payload...)
return
}

29
vendor/go.mau.fi/util/base58/doc.go vendored Normal file
View File

@@ -0,0 +1,29 @@
// Copyright (c) 2014 The btcsuite developers
// Use of this source code is governed by an ISC
// license that can be found in the LICENSE file.
/*
Package base58 provides an API for working with modified base58 and Base58Check
encodings.
# Modified Base58 Encoding
Standard base58 encoding is similar to standard base64 encoding except, as the
name implies, it uses a 58 character alphabet which results in an alphanumeric
string and allows some characters which are problematic for humans to be
excluded. Due to this, there can be various base58 alphabets.
The modified base58 alphabet used by Bitcoin, and hence this package, omits the
0, O, I, and l characters that look the same in many fonts and are therefore
hard to humans to distinguish.
# Base58Check Encoding Scheme
The Base58Check encoding scheme is primarily used for Bitcoin addresses at the
time of this writing, however it can be used to generically encode arbitrary
byte arrays into human-readable strings along with a version byte that can be
used to differentiate the same payload. For Bitcoin addresses, the extra
version is used to differentiate the network of otherwise identical public keys
which helps prevent using an address intended for one network on another.
*/
package base58

189
vendor/go.mau.fi/util/dbutil/connlog.go vendored Normal file
View File

@@ -0,0 +1,189 @@
// Copyright (c) 2022 Tulir Asokan
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
package dbutil
import (
"context"
"database/sql"
"time"
)
// LoggingExecable is a wrapper for anything with database Exec methods (i.e. sql.Conn, sql.DB and sql.Tx)
// that can preprocess queries (e.g. replacing $ with ? on SQLite) and log query durations.
type LoggingExecable struct {
UnderlyingExecable UnderlyingExecable
db *Database
}
func (le *LoggingExecable) ExecContext(ctx context.Context, query string, args ...interface{}) (sql.Result, error) {
start := time.Now()
query = le.db.mutateQuery(query)
res, err := le.UnderlyingExecable.ExecContext(ctx, query, args...)
le.db.Log.QueryTiming(ctx, "Exec", query, args, -1, time.Since(start), err)
return res, err
}
func (le *LoggingExecable) QueryContext(ctx context.Context, query string, args ...interface{}) (Rows, error) {
start := time.Now()
query = le.db.mutateQuery(query)
rows, err := le.UnderlyingExecable.QueryContext(ctx, query, args...)
le.db.Log.QueryTiming(ctx, "Query", query, args, -1, time.Since(start), err)
return &LoggingRows{
ctx: ctx,
db: le.db,
query: query,
args: args,
rs: rows,
start: start,
}, err
}
func (le *LoggingExecable) QueryRowContext(ctx context.Context, query string, args ...interface{}) *sql.Row {
start := time.Now()
query = le.db.mutateQuery(query)
row := le.UnderlyingExecable.QueryRowContext(ctx, query, args...)
le.db.Log.QueryTiming(ctx, "QueryRow", query, args, -1, time.Since(start), nil)
return row
}
func (le *LoggingExecable) Exec(query string, args ...interface{}) (sql.Result, error) {
return le.ExecContext(context.Background(), query, args...)
}
func (le *LoggingExecable) Query(query string, args ...interface{}) (Rows, error) {
return le.QueryContext(context.Background(), query, args...)
}
func (le *LoggingExecable) QueryRow(query string, args ...interface{}) *sql.Row {
return le.QueryRowContext(context.Background(), query, args...)
}
// loggingDB is a wrapper for LoggingExecable that allows access to BeginTx.
//
// While LoggingExecable has a pointer to the database and could use BeginTx, it's not technically safe since
// the LoggingExecable could be for a transaction (where BeginTx wouldn't make sense).
type loggingDB struct {
LoggingExecable
}
func (ld *loggingDB) BeginTx(ctx context.Context, opts *sql.TxOptions) (*LoggingTxn, error) {
targetDB := ld.db.RawDB
if opts != nil && opts.ReadOnly && ld.db.ReadOnlyDB != nil {
targetDB = ld.db.ReadOnlyDB
}
start := time.Now()
tx, err := targetDB.BeginTx(ctx, opts)
ld.db.Log.QueryTiming(ctx, "Begin", "", nil, -1, time.Since(start), err)
if err != nil {
return nil, err
}
return &LoggingTxn{
LoggingExecable: LoggingExecable{UnderlyingExecable: tx, db: ld.db},
UnderlyingTx: tx,
ctx: ctx,
StartTime: start,
}, nil
}
func (ld *loggingDB) Begin() (*LoggingTxn, error) {
return ld.BeginTx(context.Background(), nil)
}
type LoggingTxn struct {
LoggingExecable
UnderlyingTx *sql.Tx
ctx context.Context
StartTime time.Time
EndTime time.Time
noTotalLog bool
}
func (lt *LoggingTxn) Commit() error {
start := time.Now()
err := lt.UnderlyingTx.Commit()
lt.EndTime = time.Now()
if !lt.noTotalLog {
lt.db.Log.QueryTiming(lt.ctx, "<Transaction>", "", nil, -1, lt.EndTime.Sub(lt.StartTime), nil)
}
lt.db.Log.QueryTiming(lt.ctx, "Commit", "", nil, -1, time.Since(start), err)
return err
}
func (lt *LoggingTxn) Rollback() error {
start := time.Now()
err := lt.UnderlyingTx.Rollback()
lt.EndTime = time.Now()
if !lt.noTotalLog {
lt.db.Log.QueryTiming(lt.ctx, "<Transaction>", "", nil, -1, lt.EndTime.Sub(lt.StartTime), nil)
}
lt.db.Log.QueryTiming(lt.ctx, "Rollback", "", nil, -1, time.Since(start), err)
return err
}
type LoggingRows struct {
ctx context.Context
db *Database
query string
args []interface{}
rs Rows
start time.Time
nrows int
}
func (lrs *LoggingRows) stopTiming() {
if !lrs.start.IsZero() {
lrs.db.Log.QueryTiming(lrs.ctx, "EndRows", lrs.query, lrs.args, lrs.nrows, time.Since(lrs.start), lrs.rs.Err())
lrs.start = time.Time{}
}
}
func (lrs *LoggingRows) Close() error {
err := lrs.rs.Close()
lrs.stopTiming()
return err
}
func (lrs *LoggingRows) ColumnTypes() ([]*sql.ColumnType, error) {
return lrs.rs.ColumnTypes()
}
func (lrs *LoggingRows) Columns() ([]string, error) {
return lrs.rs.Columns()
}
func (lrs *LoggingRows) Err() error {
return lrs.rs.Err()
}
func (lrs *LoggingRows) Next() bool {
hasNext := lrs.rs.Next()
if !hasNext {
lrs.stopTiming()
} else {
lrs.nrows++
}
return hasNext
}
func (lrs *LoggingRows) NextResultSet() bool {
hasNext := lrs.rs.NextResultSet()
if !hasNext {
lrs.stopTiming()
} else {
lrs.nrows++
}
return hasNext
}
func (lrs *LoggingRows) Scan(dest ...any) error {
return lrs.rs.Scan(dest...)
}

289
vendor/go.mau.fi/util/dbutil/database.go vendored Normal file
View File

@@ -0,0 +1,289 @@
// Copyright (c) 2022 Tulir Asokan
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
package dbutil
import (
"context"
"database/sql"
"fmt"
"net/url"
"regexp"
"strings"
"time"
)
type Dialect int
const (
DialectUnknown Dialect = iota
Postgres
SQLite
)
func (dialect Dialect) String() string {
switch dialect {
case Postgres:
return "postgres"
case SQLite:
return "sqlite3"
default:
return ""
}
}
func ParseDialect(engine string) (Dialect, error) {
engine = strings.ToLower(engine)
if strings.HasPrefix(engine, "postgres") || engine == "pgx" {
return Postgres, nil
} else if strings.HasPrefix(engine, "sqlite") || strings.HasPrefix(engine, "litestream") {
return SQLite, nil
} else {
return DialectUnknown, fmt.Errorf("unknown dialect '%s'", engine)
}
}
type Rows interface {
Close() error
ColumnTypes() ([]*sql.ColumnType, error)
Columns() ([]string, error)
Err() error
Next() bool
NextResultSet() bool
Scan(...any) error
}
type Scannable interface {
Scan(...interface{}) error
}
// Expected implementations of Scannable
var (
_ Scannable = (*sql.Row)(nil)
_ Scannable = (Rows)(nil)
)
type UnderlyingContextExecable interface {
ExecContext(ctx context.Context, query string, args ...interface{}) (sql.Result, error)
QueryContext(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error)
QueryRowContext(ctx context.Context, query string, args ...interface{}) *sql.Row
}
type ContextExecable interface {
ExecContext(ctx context.Context, query string, args ...interface{}) (sql.Result, error)
QueryContext(ctx context.Context, query string, args ...interface{}) (Rows, error)
QueryRowContext(ctx context.Context, query string, args ...interface{}) *sql.Row
}
type UnderlyingExecable interface {
UnderlyingContextExecable
Exec(query string, args ...interface{}) (sql.Result, error)
Query(query string, args ...interface{}) (*sql.Rows, error)
QueryRow(query string, args ...interface{}) *sql.Row
}
type Execable interface {
ContextExecable
Exec(query string, args ...interface{}) (sql.Result, error)
Query(query string, args ...interface{}) (Rows, error)
QueryRow(query string, args ...interface{}) *sql.Row
}
type Transaction interface {
Execable
Commit() error
Rollback() error
}
// Expected implementations of Execable
var (
_ UnderlyingExecable = (*sql.Tx)(nil)
_ UnderlyingExecable = (*sql.DB)(nil)
_ Execable = (*LoggingExecable)(nil)
_ Transaction = (*LoggingTxn)(nil)
_ UnderlyingContextExecable = (*sql.Conn)(nil)
)
type Database struct {
loggingDB
RawDB *sql.DB
ReadOnlyDB *sql.DB
Owner string
VersionTable string
Log DatabaseLogger
Dialect Dialect
UpgradeTable UpgradeTable
IgnoreForeignTables bool
IgnoreUnsupportedDatabase bool
}
var positionalParamPattern = regexp.MustCompile(`\$(\d+)`)
func (db *Database) mutateQuery(query string) string {
switch db.Dialect {
case SQLite:
return positionalParamPattern.ReplaceAllString(query, "?$1")
default:
return query
}
}
func (db *Database) Child(versionTable string, upgradeTable UpgradeTable, log DatabaseLogger) *Database {
if log == nil {
log = db.Log
}
return &Database{
RawDB: db.RawDB,
loggingDB: db.loggingDB,
Owner: "",
VersionTable: versionTable,
UpgradeTable: upgradeTable,
Log: log,
Dialect: db.Dialect,
IgnoreForeignTables: true,
IgnoreUnsupportedDatabase: db.IgnoreUnsupportedDatabase,
}
}
func NewWithDB(db *sql.DB, rawDialect string) (*Database, error) {
dialect, err := ParseDialect(rawDialect)
if err != nil {
return nil, err
}
wrappedDB := &Database{
RawDB: db,
Dialect: dialect,
Log: NoopLogger,
IgnoreForeignTables: true,
VersionTable: "version",
}
wrappedDB.loggingDB.UnderlyingExecable = db
wrappedDB.loggingDB.db = wrappedDB
return wrappedDB, nil
}
func NewWithDialect(uri, rawDialect string) (*Database, error) {
db, err := sql.Open(rawDialect, uri)
if err != nil {
return nil, err
}
return NewWithDB(db, rawDialect)
}
type PoolConfig struct {
Type string `yaml:"type"`
URI string `yaml:"uri"`
MaxOpenConns int `yaml:"max_open_conns"`
MaxIdleConns int `yaml:"max_idle_conns"`
ConnMaxIdleTime string `yaml:"conn_max_idle_time"`
ConnMaxLifetime string `yaml:"conn_max_lifetime"`
}
type Config struct {
PoolConfig `yaml:",inline"`
ReadOnlyPool PoolConfig `yaml:"ro_pool"`
}
func (db *Database) Close() error {
err := db.RawDB.Close()
if db.ReadOnlyDB != nil {
err2 := db.ReadOnlyDB.Close()
if err == nil {
err = fmt.Errorf("closing read-only db failed: %w", err)
} else {
err = fmt.Errorf("%w (closing read-only db also failed: %v)", err, err2)
}
}
return err
}
func (db *Database) Configure(cfg Config) error {
if err := db.configure(db.ReadOnlyDB, cfg.ReadOnlyPool); err != nil {
return err
}
return db.configure(db.RawDB, cfg.PoolConfig)
}
func (db *Database) configure(rawDB *sql.DB, cfg PoolConfig) error {
if rawDB == nil {
return nil
}
rawDB.SetMaxOpenConns(cfg.MaxOpenConns)
rawDB.SetMaxIdleConns(cfg.MaxIdleConns)
if len(cfg.ConnMaxIdleTime) > 0 {
maxIdleTimeDuration, err := time.ParseDuration(cfg.ConnMaxIdleTime)
if err != nil {
return fmt.Errorf("failed to parse max_conn_idle_time: %w", err)
}
rawDB.SetConnMaxIdleTime(maxIdleTimeDuration)
}
if len(cfg.ConnMaxLifetime) > 0 {
maxLifetimeDuration, err := time.ParseDuration(cfg.ConnMaxLifetime)
if err != nil {
return fmt.Errorf("failed to parse max_conn_idle_time: %w", err)
}
rawDB.SetConnMaxLifetime(maxLifetimeDuration)
}
return nil
}
func NewFromConfig(owner string, cfg Config, logger DatabaseLogger) (*Database, error) {
wrappedDB, err := NewWithDialect(cfg.URI, cfg.Type)
if err != nil {
return nil, err
}
wrappedDB.Owner = owner
if logger != nil {
wrappedDB.Log = logger
}
if cfg.ReadOnlyPool.MaxOpenConns > 0 {
if cfg.ReadOnlyPool.Type == "" {
cfg.ReadOnlyPool.Type = cfg.Type
}
roUri := cfg.ReadOnlyPool.URI
if roUri == "" {
uriParts := strings.Split(cfg.URI, "?")
var qs url.Values
if len(uriParts) == 2 {
var err error
qs, err = url.ParseQuery(uriParts[1])
if err != nil {
return nil, err
}
qs.Del("_txlock")
}
qs.Set("_query_only", "true")
roUri = uriParts[0] + "?" + qs.Encode()
}
wrappedDB.ReadOnlyDB, err = sql.Open(cfg.ReadOnlyPool.Type, roUri)
if err != nil {
return nil, err
}
}
err = wrappedDB.Configure(cfg)
if err != nil {
return nil, err
}
return wrappedDB, nil
}

33
vendor/go.mau.fi/util/dbutil/json.go vendored Normal file
View File

@@ -0,0 +1,33 @@
package dbutil
import (
"database/sql/driver"
"encoding/json"
"fmt"
)
// JSON is a utility type for using arbitrary JSON data as values in database Exec and Scan calls.
type JSON struct {
Data any
}
func (j JSON) Scan(i any) error {
switch value := i.(type) {
case nil:
return nil
case string:
return json.Unmarshal([]byte(value), j.Data)
case []byte:
return json.Unmarshal(value, j.Data)
default:
return fmt.Errorf("invalid type %T for dbutil.JSON.Scan", i)
}
}
func (j JSON) Value() (driver.Value, error) {
if j.Data == nil {
return nil, nil
}
v, err := json.Marshal(j.Data)
return string(v), err
}

129
vendor/go.mau.fi/util/dbutil/log.go vendored Normal file
View File

@@ -0,0 +1,129 @@
package dbutil
import (
"context"
"regexp"
"strings"
"time"
"github.com/rs/zerolog"
)
type DatabaseLogger interface {
QueryTiming(ctx context.Context, method, query string, args []interface{}, nrows int, duration time.Duration, err error)
WarnUnsupportedVersion(current, compat, latest int)
PrepareUpgrade(current, compat, latest int)
DoUpgrade(from, to int, message string, txn bool)
// Deprecated: legacy warning method, return errors instead
Warn(msg string, args ...interface{})
}
type noopLogger struct{}
var NoopLogger DatabaseLogger = &noopLogger{}
func (n noopLogger) WarnUnsupportedVersion(_, _, _ int) {}
func (n noopLogger) PrepareUpgrade(_, _, _ int) {}
func (n noopLogger) DoUpgrade(_, _ int, _ string, _ bool) {}
func (n noopLogger) Warn(msg string, args ...interface{}) {}
func (n noopLogger) QueryTiming(_ context.Context, _, _ string, _ []interface{}, _ int, _ time.Duration, _ error) {
}
type zeroLogger struct {
l *zerolog.Logger
ZeroLogSettings
}
type ZeroLogSettings struct {
CallerSkipFrame int
Caller bool
// TraceLogAllQueries specifies whether or not all queries should be logged
// at the TRACE level.
TraceLogAllQueries bool
}
func ZeroLogger(log zerolog.Logger, cfg ...ZeroLogSettings) DatabaseLogger {
return ZeroLoggerPtr(&log, cfg...)
}
func ZeroLoggerPtr(log *zerolog.Logger, cfg ...ZeroLogSettings) DatabaseLogger {
wrapped := &zeroLogger{l: log}
if len(cfg) > 0 {
wrapped.ZeroLogSettings = cfg[0]
} else {
wrapped.ZeroLogSettings = ZeroLogSettings{
CallerSkipFrame: 2, // Skip LoggingExecable.ExecContext and zeroLogger.QueryTiming
Caller: true,
}
}
return wrapped
}
func (z zeroLogger) WarnUnsupportedVersion(current, compat, latest int) {
z.l.Warn().
Int("current_version", current).
Int("oldest_compatible_version", compat).
Int("latest_known_version", latest).
Msg("Unsupported database schema version, continuing anyway")
}
func (z zeroLogger) PrepareUpgrade(current, compat, latest int) {
evt := z.l.Info().
Int("current_version", current).
Int("oldest_compatible_version", compat).
Int("latest_known_version", latest)
if current >= latest {
evt.Msg("Database is up to date")
} else {
evt.Msg("Preparing to update database schema")
}
}
func (z zeroLogger) DoUpgrade(from, to int, message string, txn bool) {
z.l.Info().
Int("from", from).
Int("to", to).
Bool("single_txn", txn).
Str("description", message).
Msg("Upgrading database")
}
var whitespaceRegex = regexp.MustCompile(`\s+`)
func (z zeroLogger) QueryTiming(ctx context.Context, method, query string, args []interface{}, nrows int, duration time.Duration, err error) {
log := zerolog.Ctx(ctx)
if log.GetLevel() == zerolog.Disabled || log == zerolog.DefaultContextLogger {
log = z.l
}
if (!z.TraceLogAllQueries || log.GetLevel() != zerolog.TraceLevel) && duration < 1*time.Second {
return
}
if nrows > -1 {
rowLog := log.With().Int("rows", nrows).Logger()
log = &rowLog
}
query = strings.TrimSpace(whitespaceRegex.ReplaceAllLiteralString(query, " "))
log.Trace().
Err(err).
Int64("duration_µs", duration.Microseconds()).
Str("method", method).
Str("query", query).
Interface("query_args", args).
Msg("Query")
if duration >= 1*time.Second {
evt := log.Warn().
Float64("duration_seconds", duration.Seconds()).
Str("method", method).
Str("query", query)
if z.Caller {
evt = evt.Caller(z.CallerSkipFrame)
}
evt.Msg("Query took long")
}
}
func (z zeroLogger) Warn(msg string, args ...interface{}) {
z.l.Warn().Msgf(msg, args...)
}

View File

@@ -0,0 +1,21 @@
-- v0 -> v3: Sample revision jump
CREATE TABLE foo (
-- only: postgres
key BIGINT PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY,
-- only: sqlite
key INTEGER PRIMARY KEY,
data JSONB NOT NULL
);
-- only: sqlite until "end only"
CREATE TRIGGER test AFTER INSERT ON foo WHEN NEW.data->>'action' = 'delete' BEGIN
DELETE FROM test WHERE key <= NEW.data->>'index';
END;
-- end only sqlite
-- only: postgres until "end only"
CREATE FUNCTION delete_data() RETURNS TRIGGER LANGUAGE plpgsql AS $$ BEGIN
DELETE FROM test WHERE key <= NEW.data->>'index';
RETURN NEW;
END $$;
-- end only postgres

View File

@@ -0,0 +1,4 @@
-- v4: Sample outside transaction
-- transaction: off
INSERT INTO foo VALUES ('meow', '{}');

View File

@@ -0,0 +1,3 @@
-- v5 (compatible with v3+): Sample backwards-compatible upgrade
INSERT INTO foo VALUES ('meow 2', '{}');

View File

@@ -0,0 +1,11 @@
CREATE TABLE foo (
key BIGINT PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY,
data JSONB NOT NULL
);
CREATE FUNCTION delete_data() RETURNS TRIGGER LANGUAGE plpgsql AS $$ BEGIN
DELETE FROM test WHERE key <= NEW.data->>'index';
RETURN NEW;
END $$;
-- end only postgres

View File

@@ -0,0 +1,10 @@
CREATE TABLE foo (
key INTEGER PRIMARY KEY,
data JSONB NOT NULL
);
CREATE TRIGGER test AFTER INSERT ON foo WHEN NEW.data->>'action' = 'delete' BEGIN
DELETE FROM test WHERE key <= NEW.data->>'index';
END;
-- end only sqlite

View File

@@ -0,0 +1 @@
INSERT INTO foo VALUES ('meow', '{}');

View File

@@ -0,0 +1 @@
INSERT INTO foo VALUES ('meow', '{}');

View File

@@ -0,0 +1 @@
INSERT INTO foo VALUES ('meow 2', '{}');

View File

@@ -0,0 +1 @@
INSERT INTO foo VALUES ('meow 2', '{}');

View File

@@ -0,0 +1,94 @@
// Copyright (c) 2023 Tulir Asokan
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
package dbutil
import (
"context"
"database/sql"
"errors"
"fmt"
"time"
"github.com/rs/zerolog"
"go.mau.fi/util/exerrors"
"go.mau.fi/util/random"
)
var (
ErrTxn = errors.New("transaction")
ErrTxnBegin = fmt.Errorf("%w: begin", ErrTxn)
ErrTxnCommit = fmt.Errorf("%w: commit", ErrTxn)
)
type contextKey int
const (
ContextKeyDatabaseTransaction contextKey = iota
ContextKeyDoTxnCallerSkip
)
func (db *Database) DoTxn(ctx context.Context, opts *sql.TxOptions, fn func(ctx context.Context) error) error {
if ctx.Value(ContextKeyDatabaseTransaction) != nil {
zerolog.Ctx(ctx).Trace().Msg("Already in a transaction, not creating a new one")
return fn(ctx)
}
log := zerolog.Ctx(ctx).With().Str("db_txn_id", random.String(12)).Logger()
start := time.Now()
defer func() {
dur := time.Since(start)
if dur > time.Second {
val := ctx.Value(ContextKeyDoTxnCallerSkip)
callerSkip := 2
if val != nil {
callerSkip += val.(int)
}
log.Warn().
Float64("duration_seconds", dur.Seconds()).
Caller(callerSkip).
Msg("Transaction took long")
}
}()
tx, err := db.BeginTx(ctx, opts)
if err != nil {
log.Trace().Err(err).Msg("Failed to begin transaction")
return exerrors.NewDualError(ErrTxnBegin, err)
}
log.Trace().Msg("Transaction started")
tx.noTotalLog = true
ctx = log.WithContext(ctx)
ctx = context.WithValue(ctx, ContextKeyDatabaseTransaction, tx)
err = fn(ctx)
if err != nil {
log.Trace().Err(err).Msg("Database transaction failed, rolling back")
rollbackErr := tx.Rollback()
if rollbackErr != nil {
log.Warn().Err(rollbackErr).Msg("Rollback after transaction error failed")
} else {
log.Trace().Msg("Rollback successful")
}
return err
}
err = tx.Commit()
if err != nil {
log.Trace().Err(err).Msg("Commit failed")
return exerrors.NewDualError(ErrTxnCommit, err)
}
log.Trace().Msg("Commit successful")
return nil
}
func (db *Database) Conn(ctx context.Context) ContextExecable {
if ctx == nil {
return db
}
txn, ok := ctx.Value(ContextKeyDatabaseTransaction).(Transaction)
if ok {
return txn
}
return db
}

218
vendor/go.mau.fi/util/dbutil/upgrades.go vendored Normal file
View File

@@ -0,0 +1,218 @@
// Copyright (c) 2023 Tulir Asokan
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
package dbutil
import (
"database/sql"
"errors"
"fmt"
)
type upgradeFunc func(Execable, *Database) error
type upgrade struct {
message string
fn upgradeFunc
upgradesTo int
compatVersion int
transaction bool
}
var ErrUnsupportedDatabaseVersion = errors.New("unsupported database schema version")
var ErrForeignTables = errors.New("the database contains foreign tables")
var ErrNotOwned = errors.New("the database is owned by")
var ErrUnsupportedDialect = errors.New("unsupported database dialect")
func (db *Database) upgradeVersionTable() error {
if compatColumnExists, err := db.ColumnExists(nil, db.VersionTable, "compat"); err != nil {
return fmt.Errorf("failed to check if version table is up to date: %w", err)
} else if !compatColumnExists {
if tableExists, err := db.TableExists(nil, db.VersionTable); err != nil {
return fmt.Errorf("failed to check if version table exists: %w", err)
} else if !tableExists {
_, err = db.Exec(fmt.Sprintf("CREATE TABLE %s (version INTEGER, compat INTEGER)", db.VersionTable))
if err != nil {
return fmt.Errorf("failed to create version table: %w", err)
}
} else {
_, err = db.Exec(fmt.Sprintf("ALTER TABLE %s ADD COLUMN compat INTEGER", db.VersionTable))
if err != nil {
return fmt.Errorf("failed to add compat column to version table: %w", err)
}
}
}
return nil
}
func (db *Database) getVersion() (version, compat int, err error) {
if err = db.upgradeVersionTable(); err != nil {
return
}
var compatNull sql.NullInt32
err = db.QueryRow(fmt.Sprintf("SELECT version, compat FROM %s LIMIT 1", db.VersionTable)).Scan(&version, &compatNull)
if errors.Is(err, sql.ErrNoRows) {
err = nil
}
if compatNull.Valid && compatNull.Int32 != 0 {
compat = int(compatNull.Int32)
} else {
compat = version
}
return
}
const (
tableExistsPostgres = "SELECT EXISTS(SELECT 1 FROM information_schema.tables WHERE table_name=$1)"
tableExistsSQLite = "SELECT EXISTS(SELECT 1 FROM sqlite_master WHERE type='table' AND tbl_name=?1)"
)
func (db *Database) TableExists(tx Execable, table string) (exists bool, err error) {
if tx == nil {
tx = db
}
switch db.Dialect {
case SQLite:
err = db.QueryRow(tableExistsSQLite, table).Scan(&exists)
case Postgres:
err = db.QueryRow(tableExistsPostgres, table).Scan(&exists)
default:
err = ErrUnsupportedDialect
}
return
}
const (
columnExistsPostgres = "SELECT EXISTS(SELECT 1 FROM information_schema.columns WHERE table_name=$1 AND column_name=$2)"
columnExistsSQLite = "SELECT EXISTS(SELECT 1 FROM pragma_table_info(?1) WHERE name=?2)"
)
func (db *Database) ColumnExists(tx Execable, table, column string) (exists bool, err error) {
if tx == nil {
tx = db
}
switch db.Dialect {
case SQLite:
err = db.QueryRow(columnExistsSQLite, table, column).Scan(&exists)
case Postgres:
err = db.QueryRow(columnExistsPostgres, table, column).Scan(&exists)
default:
err = ErrUnsupportedDialect
}
return
}
func (db *Database) tableExistsNoError(table string) bool {
exists, err := db.TableExists(nil, table)
if err != nil {
panic(fmt.Errorf("failed to check if table exists: %w", err))
}
return exists
}
const createOwnerTable = `
CREATE TABLE IF NOT EXISTS database_owner (
key INTEGER PRIMARY KEY DEFAULT 0,
owner TEXT NOT NULL
)
`
func (db *Database) checkDatabaseOwner() error {
var owner string
if !db.IgnoreForeignTables {
if db.tableExistsNoError("state_groups_state") {
return fmt.Errorf("%w (found state_groups_state, likely belonging to Synapse)", ErrForeignTables)
} else if db.tableExistsNoError("roomserver_rooms") {
return fmt.Errorf("%w (found roomserver_rooms, likely belonging to Dendrite)", ErrForeignTables)
}
}
if db.Owner == "" {
return nil
}
if _, err := db.Exec(createOwnerTable); err != nil {
return fmt.Errorf("failed to ensure database owner table exists: %w", err)
} else if err = db.QueryRow("SELECT owner FROM database_owner WHERE key=0").Scan(&owner); errors.Is(err, sql.ErrNoRows) {
_, err = db.Exec("INSERT INTO database_owner (key, owner) VALUES (0, $1)", db.Owner)
if err != nil {
return fmt.Errorf("failed to insert database owner: %w", err)
}
} else if err != nil {
return fmt.Errorf("failed to check database owner: %w", err)
} else if owner != db.Owner {
return fmt.Errorf("%w %s", ErrNotOwned, owner)
}
return nil
}
func (db *Database) setVersion(tx Execable, version, compat int) error {
_, err := tx.Exec(fmt.Sprintf("DELETE FROM %s", db.VersionTable))
if err != nil {
return err
}
_, err = tx.Exec(fmt.Sprintf("INSERT INTO %s (version, compat) VALUES ($1, $2)", db.VersionTable), version, compat)
return err
}
func (db *Database) Upgrade() error {
err := db.checkDatabaseOwner()
if err != nil {
return err
}
version, compat, err := db.getVersion()
if err != nil {
return err
}
if compat > len(db.UpgradeTable) {
if db.IgnoreUnsupportedDatabase {
db.Log.WarnUnsupportedVersion(version, compat, len(db.UpgradeTable))
return nil
}
return fmt.Errorf("%w: currently on v%d (compatible down to v%d), latest known: v%d", ErrUnsupportedDatabaseVersion, version, compat, len(db.UpgradeTable))
}
db.Log.PrepareUpgrade(version, compat, len(db.UpgradeTable))
logVersion := version
for version < len(db.UpgradeTable) {
upgradeItem := db.UpgradeTable[version]
if upgradeItem.fn == nil {
version++
continue
}
db.Log.DoUpgrade(logVersion, upgradeItem.upgradesTo, upgradeItem.message, upgradeItem.transaction)
var tx Transaction
var upgradeConn Execable
if upgradeItem.transaction {
tx, err = db.Begin()
if err != nil {
return err
}
upgradeConn = tx
} else {
upgradeConn = db
}
err = upgradeItem.fn(upgradeConn, db)
if err != nil {
return err
}
version = upgradeItem.upgradesTo
logVersion = version
err = db.setVersion(upgradeConn, version, upgradeItem.compatVersion)
if err != nil {
return err
}
if tx != nil {
err = tx.Commit()
if err != nil {
return err
}
}
}
return nil
}

View File

@@ -0,0 +1,283 @@
// Copyright (c) 2023 Tulir Asokan
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
package dbutil
import (
"bytes"
"errors"
"fmt"
"io/fs"
"path/filepath"
"regexp"
"strconv"
"strings"
)
type UpgradeTable []upgrade
func (ut *UpgradeTable) extend(toSize int) {
if cap(*ut) >= toSize {
*ut = (*ut)[:toSize]
} else {
resized := make([]upgrade, toSize)
copy(resized, *ut)
*ut = resized
}
}
func (ut *UpgradeTable) Register(from, to, compat int, message string, txn bool, fn upgradeFunc) {
if from < 0 {
from += to
}
if from < 0 {
panic("invalid from value in UpgradeTable.Register() call")
}
if compat <= 0 {
compat = to
}
upg := upgrade{message: message, fn: fn, upgradesTo: to, compatVersion: compat, transaction: txn}
if len(*ut) == from {
*ut = append(*ut, upg)
return
} else if len(*ut) < from {
ut.extend(from + 1)
} else if (*ut)[from].fn != nil {
panic(fmt.Errorf("tried to override upgrade at %d ('%s') with '%s'", from, (*ut)[from].message, upg.message))
}
(*ut)[from] = upg
}
// Syntax is either
//
// -- v0 -> v1: Message
//
// or
//
// -- v1: Message
//
// Both syntaxes may also have a compatibility notice before the colon:
//
// -- v5 (compatible with v3+): Upgrade with backwards compatibility
var upgradeHeaderRegex = regexp.MustCompile(`^-- (?:v(\d+) -> )?v(\d+)(?: \(compatible with v(\d+)\+\))?: (.+)$`)
// To disable wrapping the upgrade in a single transaction, put `--transaction: off` on the second line.
//
// -- v5: Upgrade without transaction
// -- transaction: off
// // do dangerous stuff
var transactionDisableRegex = regexp.MustCompile(`^-- transaction: (\w*)`)
func parseFileHeader(file []byte) (from, to, compat int, message string, txn bool, lines [][]byte, err error) {
lines = bytes.Split(file, []byte("\n"))
if len(lines) < 2 {
err = errors.New("upgrade file too short")
return
}
var maybeFrom int
match := upgradeHeaderRegex.FindSubmatch(lines[0])
lines = lines[1:]
if match == nil {
err = errors.New("header not found")
} else if len(match) != 5 {
err = errors.New("unexpected number of items in regex match")
} else if maybeFrom, err = strconv.Atoi(string(match[1])); len(match[1]) > 0 && err != nil {
err = fmt.Errorf("invalid source version: %w", err)
} else if to, err = strconv.Atoi(string(match[2])); err != nil {
err = fmt.Errorf("invalid target version: %w", err)
} else if compat, err = strconv.Atoi(string(match[3])); len(match[3]) > 0 && err != nil {
err = fmt.Errorf("invalid compatible version: %w", err)
} else {
err = nil
if len(match[1]) > 0 {
from = maybeFrom
} else {
from = -1
}
message = string(match[4])
txn = true
match = transactionDisableRegex.FindSubmatch(lines[0])
if match != nil {
lines = lines[1:]
if string(match[1]) != "off" {
err = fmt.Errorf("invalid value %q for transaction flag", match[1])
}
txn = false
}
}
return
}
// To limit the next line to one dialect:
//
// -- only: postgres
//
// To limit the next N lines:
//
// -- only: sqlite for next 123 lines
//
// If the single-line limit is on the second line of the file, the whole file is limited to that dialect.
var dialectLineFilter = regexp.MustCompile(`^\s*-- only: (postgres|sqlite)(?: for next (\d+) lines| until "(end) only")?`)
// Constants used to make parseDialectFilter clearer
const (
skipUntilEndTag = -1
skipNothing = 0
skipCurrentLine = 1
skipNextLine = 2
)
func (db *Database) parseDialectFilter(line []byte) (int, error) {
match := dialectLineFilter.FindSubmatch(line)
if match == nil {
return skipNothing, nil
}
dialect, err := ParseDialect(string(match[1]))
if err != nil {
return skipNothing, err
} else if dialect == db.Dialect {
// Skip the dialect filter line
return skipCurrentLine, nil
} else if bytes.Equal(match[3], []byte("end")) {
return skipUntilEndTag, nil
} else if len(match[2]) == 0 {
// Skip the dialect filter and the next line
return skipNextLine, nil
} else {
// Parse number of lines to skip, add 1 for current line
lineCount, err := strconv.Atoi(string(match[2]))
if err != nil {
return skipNothing, fmt.Errorf("invalid line count '%s': %w", match[2], err)
}
return skipCurrentLine + lineCount, nil
}
}
var endLineFilter = regexp.MustCompile(`^\s*-- end only (postgres|sqlite)$`)
func (db *Database) filterSQLUpgrade(lines [][]byte) (string, error) {
output := make([][]byte, 0, len(lines))
for i := 0; i < len(lines); i++ {
skipLines, err := db.parseDialectFilter(lines[i])
if err != nil {
return "", err
} else if skipLines > 0 {
// Current line is implicitly skipped, so reduce one here
i += skipLines - 1
} else if skipLines == skipUntilEndTag {
startedAt := i
startedAtMatch := dialectLineFilter.FindSubmatch(lines[startedAt])
for ; i < len(lines); i++ {
if match := endLineFilter.FindSubmatch(lines[i]); match != nil {
if !bytes.Equal(match[1], startedAtMatch[1]) {
return "", fmt.Errorf(`unexpected end tag %q for %q start at line %d`, string(match[0]), string(startedAtMatch[1]), startedAt)
}
break
}
}
if i == len(lines) {
return "", fmt.Errorf(`didn't get end tag matching start %q at line %d`, string(startedAtMatch[1]), startedAt)
}
} else {
output = append(output, lines[i])
}
}
return string(bytes.Join(output, []byte("\n"))), nil
}
func sqlUpgradeFunc(fileName string, lines [][]byte) upgradeFunc {
return func(tx Execable, db *Database) error {
if skip, err := db.parseDialectFilter(lines[0]); err == nil && skip == skipNextLine {
return nil
} else if upgradeSQL, err := db.filterSQLUpgrade(lines); err != nil {
panic(fmt.Errorf("failed to parse upgrade %s: %w", fileName, err))
} else {
_, err = tx.Exec(upgradeSQL)
return err
}
}
}
func splitSQLUpgradeFunc(sqliteData, postgresData string) upgradeFunc {
return func(tx Execable, database *Database) (err error) {
switch database.Dialect {
case SQLite:
_, err = tx.Exec(sqliteData)
case Postgres:
_, err = tx.Exec(postgresData)
default:
err = fmt.Errorf("unknown dialect %s", database.Dialect)
}
return
}
}
func parseSplitSQLUpgrade(name string, fs fullFS, skipNames map[string]struct{}) (from, to, compat int, message string, txn bool, fn upgradeFunc) {
postgresName := fmt.Sprintf("%s.postgres.sql", name)
sqliteName := fmt.Sprintf("%s.sqlite.sql", name)
skipNames[postgresName] = struct{}{}
skipNames[sqliteName] = struct{}{}
postgresData, err := fs.ReadFile(postgresName)
if err != nil {
panic(err)
}
sqliteData, err := fs.ReadFile(sqliteName)
if err != nil {
panic(err)
}
from, to, compat, message, txn, _, err = parseFileHeader(postgresData)
if err != nil {
panic(fmt.Errorf("failed to parse header in %s: %w", postgresName, err))
}
sqliteFrom, sqliteTo, sqliteCompat, sqliteMessage, sqliteTxn, _, err := parseFileHeader(sqliteData)
if err != nil {
panic(fmt.Errorf("failed to parse header in %s: %w", sqliteName, err))
}
if from != sqliteFrom || to != sqliteTo || compat != sqliteCompat {
panic(fmt.Errorf("mismatching versions in postgres and sqlite versions of %s: %d/%d -> %d/%d", name, from, sqliteFrom, to, sqliteTo))
} else if message != sqliteMessage {
panic(fmt.Errorf("mismatching message in postgres and sqlite versions of %s: %q != %q", name, message, sqliteMessage))
} else if txn != sqliteTxn {
panic(fmt.Errorf("mismatching transaction flag in postgres and sqlite versions of %s: %t != %t", name, txn, sqliteTxn))
}
fn = splitSQLUpgradeFunc(string(sqliteData), string(postgresData))
return
}
type fullFS interface {
fs.ReadFileFS
fs.ReadDirFS
}
var splitFileNameRegex = regexp.MustCompile(`^(.+)\.(postgres|sqlite)\.sql$`)
func (ut *UpgradeTable) RegisterFS(fs fullFS) {
ut.RegisterFSPath(fs, ".")
}
func (ut *UpgradeTable) RegisterFSPath(fs fullFS, dir string) {
files, err := fs.ReadDir(dir)
if err != nil {
panic(err)
}
skipNames := map[string]struct{}{}
for _, file := range files {
if file.IsDir() || !strings.HasSuffix(file.Name(), ".sql") {
// do nothing
} else if _, skip := skipNames[file.Name()]; skip {
// also do nothing
} else if splitName := splitFileNameRegex.FindStringSubmatch(file.Name()); splitName != nil {
from, to, compat, message, txn, fn := parseSplitSQLUpgrade(splitName[1], fs, skipNames)
ut.Register(from, to, compat, message, txn, fn)
} else if data, err := fs.ReadFile(filepath.Join(dir, file.Name())); err != nil {
panic(err)
} else if from, to, compat, message, txn, lines, err := parseFileHeader(data); err != nil {
panic(fmt.Errorf("failed to parse header in %s: %w", file.Name(), err))
} else {
ut.Register(from, to, compat, message, txn, sqlUpgradeFunc(file.Name(), lines))
}
}
}

View File

@@ -0,0 +1,33 @@
// Copyright (c) 2022 Tulir Asokan
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
package exerrors
import (
"errors"
"fmt"
)
type DualError struct {
High error
Low error
}
func NewDualError(high, low error) DualError {
return DualError{high, low}
}
func (err DualError) Is(other error) bool {
return errors.Is(other, err.High) || errors.Is(other, err.Low)
}
func (err DualError) Unwrap() error {
return err.Low
}
func (err DualError) Error() string {
return fmt.Sprintf("%v: %v", err.High, err.Low)
}

33
vendor/go.mau.fi/util/exgjson/gjson.go vendored Normal file
View File

@@ -0,0 +1,33 @@
// Copyright (c) 2022 Tulir Asokan
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
package exgjson
import (
"strings"
)
// Escaper escapes a string for use in a GJSON path.
var Escaper = strings.NewReplacer(
`\`, `\\`,
".", `\.`,
"|", `\|`,
"#", `\#`,
"@", `\@`,
"*", `\*`,
"?", `\?`)
// Path returns a GJSON path pointing at a nested object, with each provided string being a key.
func Path(path ...string) string {
var result strings.Builder
for i, part := range path {
_, _ = Escaper.WriteString(&result, part)
if i < len(path)-1 {
result.WriteRune('.')
}
}
return result.String()
}

View File

@@ -0,0 +1,86 @@
// Copyright (c) 2022 Tulir Asokan
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
package jsontime
import (
"encoding/json"
"time"
)
func UM(time time.Time) UnixMilli {
return UnixMilli{Time: time}
}
func UMInt(ts int64) UnixMilli {
return UM(time.UnixMilli(ts))
}
func UnixMilliNow() UnixMilli {
return UM(time.Now())
}
type UnixMilli struct {
time.Time
}
func (um UnixMilli) MarshalJSON() ([]byte, error) {
if um.IsZero() {
return []byte{'0'}, nil
}
return json.Marshal(um.UnixMilli())
}
func (um *UnixMilli) UnmarshalJSON(data []byte) error {
var val int64
err := json.Unmarshal(data, &val)
if err != nil {
return err
}
if val == 0 {
um.Time = time.Time{}
} else {
um.Time = time.UnixMilli(val)
}
return nil
}
func U(time time.Time) Unix {
return Unix{Time: time}
}
func UInt(ts int64) Unix {
return U(time.Unix(ts, 0))
}
func UnixNow() Unix {
return U(time.Now())
}
type Unix struct {
time.Time
}
func (u Unix) MarshalJSON() ([]byte, error) {
if u.IsZero() {
return []byte{'0'}, nil
}
return json.Marshal(u.Unix())
}
func (u *Unix) UnmarshalJSON(data []byte) error {
var val int64
err := json.Unmarshal(data, &val)
if err != nil {
return err
}
if val == 0 {
u.Time = time.Time{}
} else {
u.Time = time.Unix(val, 0)
}
return nil
}

21
vendor/go.mau.fi/util/random/bytes.go vendored Normal file
View File

@@ -0,0 +1,21 @@
// Copyright (c) 2023 Tulir Asokan
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
package random
import (
"crypto/rand"
)
// Bytes generates the given amount of random bytes using crypto/rand, and panics if it fails.
func Bytes(n int) []byte {
data := make([]byte, n)
_, err := rand.Read(data)
if err != nil {
panic(err)
}
return data
}

87
vendor/go.mau.fi/util/random/string.go vendored Normal file
View File

@@ -0,0 +1,87 @@
// Copyright (c) 2023 Tulir Asokan
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
package random
import (
"encoding/binary"
"hash/crc32"
"strings"
"unsafe"
)
const letters = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
// StringBytes generates a random string of the given length and returns it as a byte array.
func StringBytes(n int) []byte {
if n <= 0 {
return []byte{}
}
input := Bytes(n * 2)
for i := 0; i < n; i++ {
// Risk of modulo bias is only 2 in 65535, values between 0 and 65533 are uniformly distributed
input[i] = letters[binary.BigEndian.Uint16(input[i*2:])%uint16(len(letters))]
}
input = input[:n]
return input
}
// String generates a random string of the given length.
func String(n int) string {
if n <= 0 {
return ""
}
str := StringBytes(n)
return *(*string)(unsafe.Pointer(&str))
}
func base62Encode(val uint32, minWidth int) []byte {
out := make([]byte, 0, minWidth)
for val > 0 {
out = append(out, letters[val%uint32(len(letters))])
val /= 62
}
if len(out) < minWidth {
paddedOut := make([]byte, minWidth)
copy(paddedOut[minWidth-len(out):], out)
for i := 0; i < minWidth-len(out); i++ {
paddedOut[i] = '0'
}
out = paddedOut
}
return out
}
// Token generates a GitHub-style token with the given prefix, a random part, and a checksum at the end.
// The format is `prefix_random_checksum`. The checksum is always 6 characters.
func Token(namespace string, randomLength int) string {
token := make([]byte, len(namespace)+1+randomLength+1+6)
copy(token, namespace)
token[len(namespace)] = '_'
copy(token[len(namespace)+1:], StringBytes(randomLength))
token[len(namespace)+randomLength+1] = '_'
checksum := base62Encode(crc32.ChecksumIEEE(token[:len(token)-7]), 6)
copy(token[len(token)-6:], checksum)
return *(*string)(unsafe.Pointer(&token))
}
// GetTokenPrefix parses the given token generated with Token, validates the checksum and returns the prefix namespace.
func GetTokenPrefix(token string) string {
parts := strings.Split(token, "_")
if len(parts) != 3 {
return ""
}
checksum := base62Encode(crc32.ChecksumIEEE([]byte(parts[0]+"_"+parts[1])), 6)
if string(checksum) != parts[2] {
return ""
}
return parts[0]
}
// IsToken checks if the given token is a valid token generated with Token with the given namespace..
func IsToken(namespace, token string) bool {
return GetTokenPrefix(token) == namespace
}

View File

@@ -0,0 +1,53 @@
// Copyright (c) 2021 Dillon Dixon
// Copyright (c) 2023 Tulir Asokan
//
// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at http://mozilla.org/MPL/2.0/.
// Package retryafter contains a utility function for parsing the Retry-After HTTP header.
package retryafter
import (
"net/http"
"strconv"
"time"
)
var now = time.Now
// Parse parses the backoff time specified in the Retry-After header if present.
// See https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After.
//
// The second parameter is the fallback duration to use if the header is not present or invalid.
//
// Example:
//
// time.Sleep(retryafter.Parse(resp.Header.Get("Retry-After"), 5*time.Second))
func Parse(retryAfter string, fallback time.Duration) time.Duration {
if retryAfter == "" {
return fallback
} else if t, err := time.Parse(http.TimeFormat, retryAfter); err == nil {
return t.Sub(now())
} else if seconds, err := strconv.Atoi(retryAfter); err == nil {
return time.Duration(seconds) * time.Second
}
return fallback
}
// Should returns true if the given status code indicates that the request should be retried.
//
// if retryafter.Should(resp.StatusCode, true) {
// time.Sleep(retryafter.Parse(resp.Header.Get("Retry-After"), 5*time.Second))
// }
func Should(statusCode int, retryOnRateLimit bool) bool {
switch statusCode {
case http.StatusBadGateway, http.StatusServiceUnavailable, http.StatusGatewayTimeout:
return true
case http.StatusTooManyRequests:
return retryOnRateLimit
default:
return false
}
}