* Add collapse keys to notifications
* Ensure stop processing after a notification result has been added
* Simplify the collapse key to the notif reason
* Update test
* Fix tests
* build branch
* Tune notif rate limit to dramatically reduce engagement types but always deliver conversation types
* dont build branch
---------
Co-authored-by: dholms <dtholmgren@gmail.com>
* ✨ Revert labels on takedown reversal
* ✅ Add tests to cover label reversal
* ✅ Add label reversal test to bsky package
* ⚡ Move xrpc call out of db transaction
* pseudocode for sending notification to user
* add notification push token table
* lexicon codegen
* `pds` and `api` codegen
* update lexicon
* add simple function to `putNotificationPushToken` into `notification_push_token` table
* reorgnaize imports
* add unspecced `putNotificationPushToken` to pds
* add `notification-push-tokens` table to PDS
* check if token exists before adding it to db
* add endpoint and appId to PDS table
* setup notification server
* fix logic for inserting token into db
* fix NotificationServer methods by making them static
* fix merge conflicts
* add comments on how sending ntoifications work
* remove dead code
* move notifServer to AppContext
* refactor code to handle notif sending in `Indexer` service
* add additional data when sending notifs
* clean up code
* move notif logic to indexer
* add `appId` and `endpoint` optional params to putNotificationPushToken
* clone notification code to AppView
* add endpoint to register push token with app view
* disable pds and enable app view notification server
* clean up code
* simply logic to check if a token already exists
* remove NotificationServer from PDS
* remove notification-push-token table from PDS
* remove `putNotificationPushToken` endpoint
* clean up code
* let `axios` throw error if `gorush` has an error
* let `kysely` throw error if notif cannot be registered by client
* rename `registerPushNotificationEndpoint` to `registerPushNotifications`
* delete `putNotificationPushToken` from AppView
* rename putPushNotificationToken to registerPushNotification
* remove dead notification code from pds
* remove sanitizeDisplayName from NotificationSever
* move `pushNotificationEndpoint` to config
* temp add `pushNotificationEndpoint` to dev-env setup
* remove example test from feed-generation.test.ts
* add test for registerPushNotification and clean up error handling
* move notifications test to its own file
* add test for NotificationServer to check if tokens are retrieved correctly
* remove unused functions from NotificationServer
* add additional tests for NotificationServer
* add return type to getNotificationDisplayAttributes function
* remove unnecessary comments
* remove dead NotificationServer code from PDS
* clean up code to prepareNotifsToSend
* put sending notifications as part of the backgroundQueue
* log instead of throwing error if notification attributes don't exist
* remove logs
* add more tests to `notification-server.test.ts`
* show replied with text for reply notifs
* better error handling when sending notifications via backgroundQueue
* add rate limit and batching to sending notifications
* add comments to NotificationServer
* merge with main
* use redis for rate limiting instead of normal rate limits
* move `notificationBatchSize` into config
* usePrimary db in test
* hoist push notif migration to present, update model to remove endpoint
* update push notif lexicon
* pare down lex for unregistering push
* helpers for working with service endpoints from did doc
* service-authed register/unregister push notifs
* add well-known endpoint to appview
* update bsky notif service tests
* fix to batching logic, misc tidy
* batch display notifications
* colocate all notification sending logic
* tidy tests
* remove unregister token for now
* fix registerPush lexicon, make a procedure
* fix registerPush impl, test pds proxy to notif service
* fix tests, make notif server optional when not configured
* fix notif server config for bsky app service
* move notif server rate limiting in-mem for now, add sending retry
* codegen tidy
* only push notifs on commit
* build
* fix notif rate limiter check
* send notifs from users w/o a profile
* remove build
---------
Co-authored-by: dholms <dtholmgren@gmail.com>
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* allow requesting reprocessing a repo from an indexer
* partition id guardrailes
* test
* build branch
* dont set ports
* resupply port
* give port in correct spot
* dont build branch
* origin: (21 commits)
styleguide
Increase CI test matrix size (#1490)
tweak author-feed rejects test syntax
update test
format
add repost of reply, update snaps
simplify query
fix typos
include reposted replies in posts_no_replies filter
fix bad test syntax
fix actor takedown tests
use getActor, failing atm
only include media posts by post creator
Fix condition for viewing soft-deleted followers (#1485)
✨ Expose takendown profile, their follows and followers to mods (#1456)
Handle revalidation (#1474)
Handle db pool errors on appview (#1483)
Handle db client errors on appview (#1481)
v0.6.4
✨ Allow creating moderation action with a duration (#1431)
...
* origin:
Increase CI test matrix size (#1490)
Fix condition for viewing soft-deleted followers (#1485)
✨ Expose takendown profile, their follows and followers to mods (#1456)
Handle revalidation (#1474)
Handle db pool errors on appview (#1483)
Handle db client errors on appview (#1481)
* setup bsky so that writes always go to a db primary
* test bsky primary db functionality
* sprinkle db primary over moderation and view maintainer usage
* make bsky tests more strict on primary/secondary db, fixes
* maintain prev db config on appview, require primary db, make primary/replica explicit in config
* tidy
* sketch out db replicas
* wip
* support multiple tags per bsky replica, "any" tag, fallbacks and warns
* clarify tags
* use new coordinator api across routes & services
* wire-up replica tagging config
* tidy
* cleanup entroypoints and tests for db coordinator
* fix pds test
* re-hookup migrate db
* standardize env vars for db on bsky services
* tidy
* build
* fix bav api entrypoint
* support DB_POSTGRES_URL on ingester/indexer for backwards compat
* remove build
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* Add self-label schemas
* switch around array and union for self-labels
* apply self-labels to post and profile views
* test for self labels in pds
* apply self-labels to post and profile views in bsky appview
* test for self labels in bsky appview
* update pds proxied test snapshots
* Add support for self-labels to the mdoeration sdk
* Disable unknown-labeler filtering until 3P support is added
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* tighten fuzzy actor search for larger dataset
* add indexes to support post deletion on feed_item, listing user invites on invite_code
* fix migration whitespace
* temp comment migrations and build
* Revert "temp comment migrations and build"
This reverts commit 225d2c00cce1885f2d2cffc4724843eda4b01cee.
* setup redis infra for appview indexer
* barebones bsky ingester
* add ioredis to bsky
* remove some indexer functionality from bsky api
* setup for bsky indexer
* tidy
* tidy, observe basic pipeline functioning
* process messages on bsky indexer pipeline, tidy tests and lifecycle
* trim partitions when moving cursor
* simplify config for partitions
* misc fixes for redis setup in bsky tests, add namespacing
* fix pds proxy tests
* remove cursor state from indexer partitions, simplify ingester state
* tidy
* utils for testing w/ multiple indexers, fix off-by-one xtrim
* test reingesting
* test indexer repartitioning
* add entrypoints for bsky ingester and indexer, fix db schema config, api entrypoint name, tidy
* setup and test bsky ingester backpressure, add config
* tidy
* add missing test file
* tidy redis calls, add redis sentinel config
* tidy/test some utils used in bsky pipeline
* tidy bsky pipeline tests, move helpers into dev-env
* fix pds crud test
* support redis host and password config
* better loggin/observability in ingester and indexer, make build
* add bsky ingester initial cursor config
* temporarily remove migrations from indexer/ingester
* allow ingester to batch
* packages/pg becomes packages/dev-infra with some cleanup (#1402)
* packages/dev-infra/
* Extract packages/dev-infra/_common.sh and use it
* Use period instead of source because of /bin/sh
* add docs for redis test script
* fix repartition test
* add logs to debug ci
* simplify repartitioning test, remove ci logs
---------
Co-authored-by: Jerry Chen <jerry@redelm.net>
Paul Frazee
Foysal Ahamed
Ansh
Daniel Holmgren
Eric Bailey
devin ivy
Patrick Linnane