heimoshuiyu/gonic
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

feat(podcasts): add an option to purge old episodes

Browse Source 
* Record podcast episode plays in ModifiedAt field.

* Added podcast purger.
This commit is contained in:
brian-doherty
2022-11-07 17:29:53 -06:00
committed by sentriz
parent 064bd587a8
commit 85cb0feb5a
5 changed files with 144 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files

113
README.md
View File

@@ -13,30 +13,29 @@
## features ## features
- browsing by folder (keeping your full tree intact) [see here](#directory-structure) - browsing by folder (keeping your full tree intact) [see here](#directory-structure)
- browsing by tags (using [taglib](https://taglib.org/) - supports mp3, opus, flac, ape, m4a, wav, etc.) - browsing by tags (using [taglib](https://taglib.org/) - supports mp3, opus, flac, ape, m4a, wav, etc.)
- on-the-fly audio transcoding and caching (requires [ffmpeg](https://ffmpeg.org/)) (thank you [spijet](https://github.com/spijet/)) - on-the-fly audio transcoding and caching (requires [ffmpeg](https://ffmpeg.org/)) (thank you [spijet](https://github.com/spijet/))
- jukebox mode (thank you [lxea](https://github.com/lxea/)) - jukebox mode (thank you [lxea](https://github.com/lxea/))
- support for podcasts (thank you [lxea](https://github.com/lxea/)) - support for podcasts (thank you [lxea](https://github.com/lxea/))
- pretty fast scanning (with my library of ~27k tracks, initial scan takes about 10m, and about 5s after incrementally) - pretty fast scanning (with my library of ~27k tracks, initial scan takes about 10m, and about 5s after incrementally)
- multiple users, each with their own transcoding preferences, playlists, top tracks, top artists, etc. - multiple users, each with their own transcoding preferences, playlists, top tracks, top artists, etc.
- [last.fm](https://www.last.fm/) scrobbling - [last.fm](https://www.last.fm/) scrobbling
- [listenbrainz](https://listenbrainz.org/) scrobbling (thank you [spezifisch](https://github.com/spezifisch), [lxea](https://github.com/lxea)) - [listenbrainz](https://listenbrainz.org/) scrobbling (thank you [spezifisch](https://github.com/spezifisch), [lxea](https://github.com/lxea))
- artist similarities and biographies from the last.fm api - artist similarities and biographies from the last.fm api
- multiple genre support (see `GONIC_GENRE_SPLIT` to split tag strings on a character, eg. `;`, and browse them individually) - multiple genre support (see `GONIC_GENRE_SPLIT` to split tag strings on a character, eg. `;`, and browse them individually)
- a web interface for configuration (set up last.fm, manage users, start scans, etc.) - a web interface for configuration (set up last.fm, manage users, start scans, etc.)
- support for the [album-artist](https://mkoby.com/2007/02/18/artist-versus-album-artist/) tag, to not clutter your artist list with compilation album appearances - support for the [album-artist](https://mkoby.com/2007/02/18/artist-versus-album-artist/) tag, to not clutter your artist list with compilation album appearances
- written in [go](https://golang.org/), so lightweight and suitable for a raspberry pi, etc. (see ARM images below) - written in [go](https://golang.org/), so lightweight and suitable for a raspberry pi, etc. (see ARM images below)
- newer salt and token auth - newer salt and token auth
- tested on [symfonium](https://symfonium.app), [dsub](https://f-droid.org/en/packages/github.daneren2005.dsub/), [jamstash](http://jamstash.com/), [sublime music](https://gitlab.com/sublime-music/sublime-music/), [soundwaves](https://apps.apple.com/us/app/soundwaves/id736139596), and [stmp](https://github.com/wildeyedskies/stmp) - tested on [symfonium](https://symfonium.app), [dsub](https://f-droid.org/en/packages/github.daneren2005.dsub/), [jamstash](http://jamstash.com/), [sublime music](https://gitlab.com/sublime-music/sublime-music/), [soundwaves](https://apps.apple.com/us/app/soundwaves/id736139596), and [stmp](https://github.com/wildeyedskies/stmp)
## installation ## installation
the default login is **admin**/**admin**. the default login is **admin**/**admin**.
password can then be changed from the web interface password can then be changed from the web interface
### ...from source ### ...from source
```bash ```bash
$ apt install build-essential git sqlite libtag1-dev ffmpeg libasound-dev # for debian like $ apt install build-essential git sqlite libtag1-dev ffmpeg libasound-dev # for debian like
@@ -47,13 +46,14 @@ $ gonic -h # or see "configuration options below"
``` ```
**note:** unfortunately if you do this above, you'll be compiling gonic locally on your machine **note:** unfortunately if you do this above, you'll be compiling gonic locally on your machine
(if someone knows how I can statically link sqlite3 and taglib, please let me know so I can distribute static binaries) (if someone knows how I can statically link sqlite3 and taglib, please let me know so I can distribute static binaries)
### ...with docker ### ...with docker
the image is available on dockerhub as [sentriz/gonic](https://hub.docker.com/r/sentriz/gonic) the image is available on dockerhub as [sentriz/gonic](https://hub.docker.com/r/sentriz/gonic)
available architectures are available architectures are
- `linux/amd64` - `linux/amd64`
- `linux/arm/v6` - `linux/arm/v6`
- `linux/arm/v7` - `linux/arm/v7`
@@ -62,31 +62,31 @@ available architectures are
```yaml ```yaml
# example docker-compose.yml # example docker-compose.yml
version: '2.4' version: "2.4"
services: services:
gonic: gonic:
image: sentriz/gonic:latest image: sentriz/gonic:latest
environment: environment:
- TZ - TZ
# optionally, see more env vars below # optionally, see more env vars below
expose: expose:
- 80 - 80
volumes: volumes:
- ./data:/data # gonic db etc - ./data:/data # gonic db etc
- /path/to/music:/music:ro # your music - /path/to/music:/music:ro # your music
- /path/to/podcasts:/podcasts # your podcasts - /path/to/podcasts:/podcasts # your podcasts
- /path/to/cache:/cache # transcode / covers / etc cache dir - /path/to/cache:/cache # transcode / covers / etc cache dir
# set the following two sections if you've enabled jukebox # set the following two sections if you've enabled jukebox
group_add: group_add:
- audio - audio
devices: devices:
- /dev/snd:/dev/snd - /dev/snd:/dev/snd
``` ```
then start with `docker-compose up -d` then start with `docker-compose up -d`
### ...with systemd ### ...with systemd
tested on Ubuntu 21.04 tested on Ubuntu 21.04
@@ -141,58 +141,62 @@ $ journalctl --follow --unit gonic # check logs
``` ```
should be installed and running on boot now 👍 should be installed and running on boot now 👍
view the admin UI at http://localhost:4747 view the admin UI at <http://localhost:4747>
### ...elsewhere ### ...elsewhere
[![](https://repology.org/badge/vertical-allrepos/gonic.svg)](https://repology.org/project/gonic/versions) [![](https://repology.org/badge/vertical-allrepos/gonic.svg)](https://repology.org/project/gonic/versions)
## configuration options ## configuration options
| env var | command line arg | description | | env var | command line arg | description |
| ----------------------- | ------------------ | ----------------------------------------------------------------------------------------------------------- | | ------------------------- | ----------------------- | ----------------------------------------------------------------------------------------------------------- |
| `GONIC_MUSIC_PATH` | `-music-path` | path to your music collection (see also multi-folder support below) | | `GONIC_MUSIC_PATH` | `-music-path` | path to your music collection (see also multi-folder support below) |
| `GONIC_PODCAST_PATH` | `-podcast-path` | path to a podcasts directory | | `GONIC_PODCAST_PATH` | `-podcast-path` | path to a podcasts directory |
| `GONIC_CACHE_PATH` | `-cache-path` | path to store audio transcodes, covers, etc | | `GONIC_CACHE_PATH` | `-cache-path` | path to store audio transcodes, covers, etc |
| `GONIC_DB_PATH` | `-db-path` | **optional** path to database file | | `GONIC_DB_PATH` | `-db-path` | **optional** path to database file |
| `GONIC_HTTP_LOG` | `-http-log` | **optional** http request logging, enabled by default | | `GONIC_HTTP_LOG` | `-http-log` | **optional** http request logging, enabled by default |
| `GONIC_LISTEN_ADDR` | `-listen-addr` | **optional** host and port to listen on (eg. `0.0.0.0:4747`, `127.0.0.1:4747`) (_default_ `0.0.0.0:4747`) | | `GONIC_LISTEN_ADDR` | `-listen-addr` | **optional** host and port to listen on (eg. `0.0.0.0:4747`, `127.0.0.1:4747`) (_default_ `0.0.0.0:4747`) |
| `GONIC_TLS_CERT` | `-tls-cert` | **optional** path to a TLS cert (enables HTTPS listening) | | `GONIC_TLS_CERT` | `-tls-cert` | **optional** path to a TLS cert (enables HTTPS listening) |
| `GONIC_TLS_KEY` | `-tls-key` | **optional** path to a TLS key (enables HTTPS listening) | | `GONIC_TLS_KEY` | `-tls-key` | **optional** path to a TLS key (enables HTTPS listening) |
| `GONIC_PROXY_PREFIX` | `-proxy-prefix` | **optional** url path prefix to use if behind reverse proxy. eg `/gonic` (see example configs below) | | `GONIC_PROXY_PREFIX` | `-proxy-prefix` | **optional** url path prefix to use if behind reverse proxy. eg `/gonic` (see example configs below) |
| `GONIC_SCAN_INTERVAL` | `-scan-interval` | **optional** interval (in minutes) to check for new music (automatic scanning disabled if omitted) | | `GONIC_SCAN_INTERVAL` | `-scan-interval` | **optional** interval (in minutes) to check for new music (automatic scanning disabled if omitted) |
| `GONIC_SCAN_WATCHER ` | `-scan-watcher-enabled` | **optional** whether to watch file system for new music and rescan | | `GONIC_SCAN_WATCHER` | `-scan-watcher-enabled` | **optional** whether to watch file system for new music and rescan |
| `GONIC_JUKEBOX_ENABLED` | `-jukebox-enabled` | **optional** whether the subsonic [jukebox api](https://airsonic.github.io/docs/jukebox/) should be enabled | | `GONIC_JUKEBOX_ENABLED` | `-jukebox-enabled` | **optional** whether the subsonic [jukebox api](https://airsonic.github.io/docs/jukebox/) should be enabled |
| `GONIC_GENRE_SPLIT` | `-genre-split` | **optional** a string or character to split genre tags on for multi-genre support (eg. `;`) | | `GONIC_PODCAST_PURGE_AGE` | `-podcast-purge-age` | **optional** age (in days) to purge podcast episodes if not accessed |
| `GONIC_GENRE_SPLIT` | `-genre-split` | **optional** a string or character to split genre tags on for multi-genre support (eg. `;`) |
## screenshots ## screenshots
|||||| | | | | | |
|:-:|:-:|:-:|:-:|:-:| | :-----------------------------------------------------------------------------: | :-----------------------------------------------------------------------------: | :-----------------------------------------------------------------------------: | :-----------------------------------------------------------------------------: | :-----------------------------------------------------------------------------: |
![](https://raw.githubusercontent.com/sentriz/gonic/master/.github/scrot_1.png)|![](https://raw.githubusercontent.com/sentriz/gonic/master/.github/scrot_2.png)|![](https://raw.githubusercontent.com/sentriz/gonic/master/.github/scrot_3.png)|![](https://raw.githubusercontent.com/sentriz/gonic/master/.github/scrot_4.png)|![](https://raw.githubusercontent.com/sentriz/gonic/master/.github/scrot_5.png)| | ![](https://raw.githubusercontent.com/sentriz/gonic/master/.github/scrot_1.png) | ![](https://raw.githubusercontent.com/sentriz/gonic/master/.github/scrot_2.png) | ![](https://raw.githubusercontent.com/sentriz/gonic/master/.github/scrot_3.png) | ![](https://raw.githubusercontent.com/sentriz/gonic/master/.github/scrot_4.png) | ![](https://raw.githubusercontent.com/sentriz/gonic/master/.github/scrot_5.png) |
## multiple folders support (v0.15+) ## multiple folders support (v0.15+)
gonic supports multiple music folders. this can be handy if you have your music separated by albums, compilations, singles. or maybe 70s, 80s, 90s. whatever. gonic supports multiple music folders. this can be handy if you have your music separated by albums, compilations, singles. or maybe 70s, 80s, 90s. whatever.
if you're running gonic with the command line, stack the `-music-path` arg if you're running gonic with the command line, stack the `-music-path` arg
```shell ```shell
$ gonic -music-path /path/to/albums -music-path /path/to/compilations $ gonic -music-path /path/to/albums -music-path /path/to/compilations
``` ```
if you're running gonic with ENV_VARS, or docker, try separate with a comma if you're running gonic with ENV_VARS, or docker, try separate with a comma
```shell ```shell
GONIC_MUSIC_PATH=/path/to/albums,/path/to/compilations GONIC_MUSIC_PATH=/path/to/albums,/path/to/compilations
``` ```
if you're running gonic with the config file, you can repeat the `music-path` option if you're running gonic with the config file, you can repeat the `music-path` option
```shell ```shell
music-path /path/to/albums music-path /path/to/albums
music-path /path/to/compilations music-path /path/to/compilations
``` ```
after that, most subsonic clients should allow you to select which music folder to use. after that, most subsonic clients should allow you to select which music folder to use.
queries like show me "recently played compilations" or "recently added albums" are possible for example. queries like show me "recently played compilations" or "recently added albums" are possible for example.
## example nginx config with `GONIC_PROXY_PREFIX` ## example nginx config with `GONIC_PROXY_PREFIX`
@@ -208,11 +212,12 @@ queries like show me "recently played compilations" or "recently added albums" a
## directory structure ## directory structure
when browsing by folder, any arbitrary and nested folder layout is supported, with the following caveats: when browsing by folder, any arbitrary and nested folder layout is supported, with the following caveats:
- Files from the same album must all be in the same folder - Files from the same album must all be in the same folder
- All files in a folder must be from the same album - All files in a folder must be from the same album
please see [here](https://github.com/sentriz/gonic/issues/89) for more context please see [here](https://github.com/sentriz/gonic/issues/89) for more context
``` ```
music music

11
cmd/gonic/gonic.go
View File

@@ -1,4 +1,5 @@
// Package main is the gonic server entrypoint // Package main is the gonic server entrypoint
//
//nolint:lll // flags help strings //nolint:lll // flags help strings
package main package main
@@ -35,9 +36,10 @@ func main() {
confPodcastPath := set.String("podcast-path", "", "path to podcasts") confPodcastPath := set.String("podcast-path", "", "path to podcasts")
confCachePath := set.String("cache-path", "", "path to cache") confCachePath := set.String("cache-path", "", "path to cache")
confDBPath := set.String("db-path", "gonic.db", "path to database (optional)") confDBPath := set.String("db-path", "gonic.db", "path to database (optional)")
confScanInterval := set.Int("scan-interval", 0, "interval (in minutes) to automatically scan music (optional)") confScanIntervalMins := set.Int("scan-interval", 0, "interval (in minutes) to automatically scan music (optional)")
confScanWatcher := set.Bool("scan-watcher-enabled", false, "whether to watch file system for new music and rescan (optional)") confScanWatcher := set.Bool("scan-watcher-enabled", false, "whether to watch file system for new music and rescan (optional)")
confJukeboxEnabled := set.Bool("jukebox-enabled", false, "whether the subsonic jukebox api should be enabled (optional)") confJukeboxEnabled := set.Bool("jukebox-enabled", false, "whether the subsonic jukebox api should be enabled (optional)")
confPodcastPurgeAgeDays := set.Int("podcast-purge-age", 0, "age (in days) to purge podcast episodes if not accessed (optional)")
confProxyPrefix := set.String("proxy-prefix", "", "url path prefix to use if behind proxy. eg '/gonic' (optional)") confProxyPrefix := set.String("proxy-prefix", "", "url path prefix to use if behind proxy. eg '/gonic' (optional)")
confGenreSplit := set.String("genre-split", "\n", "character or string to split genre tag data on (optional)") confGenreSplit := set.String("genre-split", "\n", "character or string to split genre tag data on (optional)")
confHTTPLog := set.Bool("http-log", true, "http request logging (optional)") confHTTPLog := set.Bool("http-log", true, "http request logging (optional)")
@@ -131,8 +133,8 @@ func main() {
g.Add(server.StartHTTP(*confListenAddr, *confTLSCert, *confTLSKey)) g.Add(server.StartHTTP(*confListenAddr, *confTLSCert, *confTLSKey))
g.Add(server.StartSessionClean(cleanTimeDuration)) g.Add(server.StartSessionClean(cleanTimeDuration))
g.Add(server.StartPodcastRefresher(time.Hour)) g.Add(server.StartPodcastRefresher(time.Hour))
if *confScanInterval > 0 { if *confScanIntervalMins > 0 {
tickerDur := time.Duration(*confScanInterval) * time.Minute tickerDur := time.Duration(*confScanIntervalMins) * time.Minute
g.Add(server.StartScanTicker(tickerDur)) g.Add(server.StartScanTicker(tickerDur))
} }
if *confScanWatcher { if *confScanWatcher {
@@ -141,6 +143,9 @@ func main() {
if *confJukeboxEnabled { if *confJukeboxEnabled {
g.Add(server.StartJukebox()) g.Add(server.StartJukebox())
} }
if *confPodcastPurgeAgeDays > 0 {
g.Add(server.StartPodcastPurger(time.Duration(*confPodcastPurgeAgeDays) * 24 * time.Hour))
}
if err := g.Run(); err != nil { if err := g.Run(); err != nil {
log.Panicf("error in job: %v", err) log.Panicf("error in job: %v", err)

25
podcasts/podcasts.go
View File

@@ -516,6 +516,31 @@ func (p *Podcasts) DeletePodcastEpisode(podcastEpisodeID int) error {
return err return err
} }
func (p *Podcasts) PurgeOldPodcasts(maxAge time.Duration) error {
expDate := time.Now().Add(-maxAge)
var episodes []*db.PodcastEpisode
err := p.db.
Debug().
Where("created_at < ?", expDate).
Where("updated_at < ?", expDate).
Where("modified_at < ?", expDate).
Find(&episodes).
Error
if err != nil && !errors.Is(err, gorm.ErrRecordNotFound) {
return fmt.Errorf("find podcasts: %w", err)
}
for _, episode := range episodes {
episode.Status = db.PodcastEpisodeStatusDeleted
if err := p.db.Save(episode).Error; err != nil {
return fmt.Errorf("save new podcast status: %w", err)
}
if err := os.Remove(filepath.Join(p.baseDir, episode.Path)); err != nil {
return fmt.Errorf("remove podcast path: %w", err)
}
}
return nil
}
func pathSafe(in string) string { func pathSafe(in string) string {
return filepath.Clean(strings.ReplaceAll(in, string(filepath.Separator), "_")) return filepath.Clean(strings.ReplaceAll(in, string(filepath.Separator), "_"))
} }

28
server/ctrlsubsonic/handlers_raw.go
View File

@@ -106,6 +106,24 @@ func streamUpdateStats(dbc *db.DB, userID, albumID int, playTime time.Time) erro
return nil return nil
} }
func streamUpdatePodcastEpisodeStats(dbc *db.DB, peID int) error {
var pe db.PodcastEpisode
err := dbc.
Where("id=?", peID).
First(&pe).
Error
if err != nil && !errors.Is(err, gorm.ErrRecordNotFound) {
return fmt.Errorf("find podcast episode: %w", err)
}
pe.ModifiedAt = time.Now()
if err := dbc.Save(&pe).Error; err != nil {
return fmt.Errorf("save podcast episode: %w", err)
}
return nil
}
const ( const (
coverDefaultSize = 600 coverDefaultSize = 600
coverCacheFormat = "png" coverCacheFormat = "png"
@@ -271,7 +289,15 @@ func (c *Controller) ServeStream(w http.ResponseWriter, r *http.Request) *spec.R
if track, ok := file.(*db.Track); ok && track.Album != nil { if track, ok := file.(*db.Track); ok && track.Album != nil {
defer func() { defer func() {
if err := streamUpdateStats(c.DB, user.ID, track.Album.ID, time.Now()); err != nil { if err := streamUpdateStats(c.DB, user.ID, track.Album.ID, time.Now()); err != nil {
log.Printf("error updating status: %v", err) log.Printf("error updating track status: %v", err)
}
}()
}
if pe, ok := file.(*db.PodcastEpisode); ok {
defer func() {
if err := streamUpdatePodcastEpisodeStats(c.DB, pe.ID); err != nil {
log.Printf("error updating podcast episode status: %v", err)
} }
}() }()
} }

25
server/server.go
View File

@@ -389,6 +389,31 @@ func (s *Server) StartPodcastRefresher(dur time.Duration) (FuncExecute, FuncInte
} }
} }
func (s *Server) StartPodcastPurger(maxAge time.Duration) (FuncExecute, FuncInterrupt) {
ticker := time.NewTicker(24 * time.Hour)
done := make(chan struct{})
waitFor := func() error {
for {
select {
case <-done:
return nil
case <-ticker.C:
if err := s.podcast.PurgeOldPodcasts(maxAge); err != nil {
log.Printf("error purging old podcasts: %v", err)
}
}
}
}
return func() error {
log.Printf("starting job 'podcast purger'\n")
return waitFor()
}, func(_ error) {
// stop job
ticker.Stop()
done <- struct{}{}
}
}
func (s *Server) StartSessionClean(dur time.Duration) (FuncExecute, FuncInterrupt) { func (s *Server) StartSessionClean(dur time.Duration) (FuncExecute, FuncInterrupt) {
ticker := time.NewTicker(dur) ticker := time.NewTicker(dur)
done := make(chan struct{}) done := make(chan struct{})