* mv appview
* copy
* finalize copy
* package names
* big WIP
* first pass at mod servce
* some tidy
* tidy & fix compiler errors
* rename to ozone, db migrations, add to dev-env & pds cfg
* getRecord & getRepo mostly working
* fix open handle
* get record tests all working
* moderation events working
* statuses working
* tidy test suite
* search repos
* server & db tests
* moderation tests
* wip daemon + push events
* pds fanout working
* fix db test
* fanning takedowns out to appview
* rm try/catch
* bsky moderation test
* introduce mod subject wrappers
* more tidy
* refactor event reversal
* tidy some db stuff
* tidy
* rename service to mod-service
* fix test
* tidy config
* refactor auth in bsky
* wip patching up auto-mod
* add label ingester in appview
* fix a couple build issues
* fix some timing bugs
* tidy polling logic
* fix up tests
* fix some pds tests
* eslint ignore
* fix ozone tests
* move seeds to dev-env
* move images around
* fix db schemas
* use service auth admin reqs
* fix remaining tests
* auth tests bsky
* another test
* random tidy
* fix up search
* clean up bsky mod service
* more tidy
* default attempts to 0
* tidy old test
* random tidy
* tidy package.json
* tidy logger
* takedownId -> takedownRef
* misc pr feedback
* split daemon out from ozone application
* fix blob takedown mgiration
* refactor ozone config
* do push event fanout on write instead of on read
* make suspend error work again
* add attempts check & add supporting index
* fix takedown test ref
* get tests working
* rm old test
* fix timing bug in event pusher tests
* attempt another fix for timing bug
* await req
* service files
* remove labelerDid cfg
* update snaps for labeler did + some cfg changes
* fix more snaps
* pnpm i
* build ozone images
* build
* make label provider optional
* fix build issues
* fix build
* fix build
* build pds
* build on ghcr
* fix syntax in entry
* another fix
* use correct import
* export logger
* remove event reverser
* adjust push event fanout
* push out multiple
* remove builds
* 🚧 WIP with proposed lexicons for event based mod architecture
* 🚧 Remove unnecessary moderation action lexicon
* 🚧 Working on event based actions
* ✨ Add escalated subject status
* 🐛 Alright, fixed the error in lexicon
* 🚧 Working through reversal
* ✨ Cleanup build errors
* ✨ Add subject status endpoint
* ✨ Add handler
* ✨ get reports from mod actions table
* :rightwards_twisted_arrows: Merge with upstream
* 🚧 Builds but test network doesnt start
* ✨ Tests passing on event based status change
* ✨ Rename index
* ♻️ Rename takeModerationAction->emitModerationEvent
* ✨ Implement label reversal
* ✅ Auto-revert test working
* ♻️✅ Refactored to event types and tests are passing
* ✨ Add takedown event sequence validation
* ✨ Adds support for blobCid status
* 🧹 Cleanup unnecessary method:
* ✨ Hydrate handles with status and events
* ✨ Re-implement auto reversal
* ✨ Add takendown and mute filters
* ✨ Allow filtering events by type
* ✨ Allow filtering events by creator did
* ✨ Add subjectStatus to record and repoview
* ✨ Add persistent note feature
* ✨ Log send email event
* 🐛 Fix logging send email event
* ✨ Better type
* ✨ Adjust migration to create separate moderation_event table
* 🧹 Cleanup types
* ✅ Adjust tests with mod event emitter
* ✨ Fix more tests around takedowns
* ✅ Get test suite to pass
* ✅ Get test suite to pass for pds
* ✅ Get test suite to pass for pds
* ✅ Update snapshot for feedgen
* ✅ Why are more snapshots updating?
* ♻️ Rename getModerationEvents -> queryModerationEvents
* ♻️ Rename getModerationStatuses -> queryModerationStatuses
* ♻️ Rename persistNote->sticky
* 🐛 Rename subject
* ♻️ Cleanup expiresAt for scheduled actions
* ✨ Add more tests, allow fetching mod history for all content by a user
* ✅ Fix repo and record tests
* ✨ Migrate reports and actions to events
* 🐛 Fix escalated status overwrite
* ✨ Implement direct sql query to create events from actions and reports
* 🚧 Adding keyset pagination for subject statuses
* ✨ Add migration for lastReportedAt
* ✨ Migrate blob cids
* ✨ Fix pagination on mod subject list endpoint
* 🐛 Fix blob actions
* ✅ All tests passing on bsky package
* ✅ Bring back snapshots
* ✅ Skipping timeline test temporarily
* ✅ Skipping some more tests to isolate failing ones
* ✅ Bring back list-feed test
* ✅ Bring back timeline test
* ✅ Fix label action in seeding
* ✅ Enable timeline proxied test
* ✅ Enable search actor proxied test
* ✅ Enable feedgen tests
* ✅ Fix test for admin/get-record
* ✨ Move note to comment for subject status
* ✨ Accept comments in mute event
* ✨ Remap flag event to ack event
* 🐛 Add legacyRef in report union selection
* @atproto/api 0.6.24-next.0
* @atproto/api 0.6.24-next.1
* ✨ Adjust migration export and add index for blobCids column
* ✨ Maintin action ids when migrating
* ✨ Paginate events using createdAt timestamp
* ✅ Update snapshot for pds test with events cursor update
* ✅ Use only events for snapshot testing
* ✅ Use only events for snapshot in the remaining test
* relative paths to lexicons for build
* fix bsky periodic event reversal in service entrypoint
* ✨ Allow comments in takedown and label
* ✨ Only import reports on consecutive run of the migration script
* ✨ Adjust moderation property of blob entries
* determine latest reports to migrate
* ✨ Process new reports for subject status
* ✨ Process unresolved reports on first migration run
* fix transaction error, process just unresolved reports, make reported-at updates safe for reruns
* tidy
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* rm tables
* rm event-stream & proxied
* Remove appview services, move label service to pds
* only proxy appview stuff
* delete more tables
* Start removing message dispatched from pds
* more syncing-up removal of message dispatcher in pds
* merged
* remove feedgens from pds, remove getPopular
* remove unused image helper from pds
* fixing compiler errors
* clean up sharp
* rm label service
* first pass on cleaning up tests
* fix up a bunch of tests
* moderation view tests
* last admin tests
* got a lil overzealous in deletes
* clean up unused cfg
* clean up label table
* simplify admin repo search query/logic
* tidy pds entrypoint
* in-progress pds config changes
* cfg fiddling
* finish cleaning up cfg/ctx
* comments
* building
* pds prefix on env
* test env
* collapse pds migrations down into a single migration
* fix up dev-env
* tidy
* cleanup
* fix pds admin tests
* fix handle test
* fix pds proxy tests
* fix subscribe repos test
* fix sqlite config in pds tests
* add sqlite clause in sequencer-leader
* fix actor search w/ sqlite on pds
* fixes
* fix dev env build
* update pds service entrypoint
* simple env example
* make takedown ids opaque identifiers in the pds
* use pds routes for api tests
* update pds dockerfile with volume and correct port env var
* add a couple env vars to example
* add comments to env example
* @atproto/pds 0.2.0-beta.0
* @atproto/aws 0.0.1-beta.0
* appview did
* @atproto/aws 0.0.1
* enable logs by default
* update env example
* bugfixing sandbox issues
* consistency in pds env var name for appview url
* log on pds start and stop, configure version at runtime
* @atproto/pds 0.2.0-beta.1
* fix semver matching for pds beta version
* v0.2.0-beta.2
* default invites to being not required
* fix flaky test
* limit db connections in tests
* publish 0.2.0-beta.d3
* fix invite required parsing
* @atproto/pds 0.2.0-beta.5
* Proxy getPopularFeedGenerators on simplified pds (#1222)
proxy getPopularFeedGenerators on pds
Co-authored-by: dholms <dtholmgren@gmail.com>
* tidy migrations
* fix service entry
* bump version
* change auth order
* bump version
* bump version
* add upgradeRepoVersion & fallback url for cdn
* bump version
* merging
* merge pds
* building dev-env
* merging tests
* merge service entry
* test fixing
* tidy
* fix admin search
* tidy
* tidy
* add snap for getListFeed
* add backup nameserver cfg
* tidy + pr feedback
* tidy
* tidy env
* bit more
* re-add dotenv to root package.json
* fix dep
* build branch
* fix tests
* Refactor tests to make better use of dev-env (#1690)
* refactor pds tests to use dev env
* refactor bsky tests
* fix pds test
* tidy bsky tests
* build pds correctly
* fix entry point
* default logging to false (for now)
* format service entry
* Switch takedown ids back to ints on pds distribution (#1694)
* switch takedown ids back to ints, consistent with live pds
* tidy/fix migration
* update migration for sqlite
* export moderation action reversal
* takedown tests
* dont build branch
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* setup redis infra for appview indexer
* barebones bsky ingester
* add ioredis to bsky
* remove some indexer functionality from bsky api
* setup for bsky indexer
* tidy
* tidy, observe basic pipeline functioning
* process messages on bsky indexer pipeline, tidy tests and lifecycle
* trim partitions when moving cursor
* simplify config for partitions
* misc fixes for redis setup in bsky tests, add namespacing
* fix pds proxy tests
* remove cursor state from indexer partitions, simplify ingester state
* tidy
* utils for testing w/ multiple indexers, fix off-by-one xtrim
* test reingesting
* test indexer repartitioning
* add entrypoints for bsky ingester and indexer, fix db schema config, api entrypoint name, tidy
* setup and test bsky ingester backpressure, add config
* tidy
* add missing test file
* tidy redis calls, add redis sentinel config
* tidy/test some utils used in bsky pipeline
* tidy bsky pipeline tests, move helpers into dev-env
* fix pds crud test
* support redis host and password config
* better loggin/observability in ingester and indexer, make build
* add bsky ingester initial cursor config
* temporarily remove migrations from indexer/ingester
* allow ingester to batch
* packages/pg becomes packages/dev-infra with some cleanup (#1402)
* packages/dev-infra/
* Extract packages/dev-infra/_common.sh and use it
* Use period instead of source because of /bin/sh
* add docs for redis test script
* fix repartition test
* add logs to debug ci
* simplify repartitioning test, remove ci logs
---------
Co-authored-by: Jerry Chen <jerry@redelm.net>
* Don't serve high-level details about taken-down feedgens
* Takedown checks on sync routes
* Use simpler check for takedowns on sync routes
* ensure takendown feedgens are handled in bsky appview
* Model post and profile aggs in bsky
* Add background, onCommit, db and background stats
* Add aggregations for post and profile
* Use background queue in bsky labeler, misc plumbing
* Process background throughout bsky tests
* bsky tidy and fixes
* Build views using agg tables in bsky
* Views of feedgens on bsky
* Setup custom feeds on bsky
* Feed views refactor for bsky, implement getFeed w/ auth passthrough
* Passthrough auth from bsky appview to feedgen, fixes/tidy/tests
* Test custom algos on bsky
* Test pds getFeed proxy
* tidy, fix build
* tidy
* Model post and profile aggs in bsky
* Add background, onCommit, db and background stats
* Add aggregations for post and profile
* Use background queue in bsky labeler, misc plumbing
* Process background throughout bsky tests
* bsky tidy and fixes
* Build views using agg tables in bsky
* Views of feedgens on bsky
* tidy, fix build
* tidy/fix bsky migration