* 🚧 Working through an nullable review state
* ✅ Update snapshots on some tests
* ✅ Update snapshots on some tests
* ✅ Add test for reviewOptional status mutation
* ✨ Add divertBlobs event to send blobs to abyss
* ♻️ Rename reviewOptional -> reviewNone
* ♻️ Rename modEventDivertBlobs -> modEventDivert
* 🐛 Rename event type checker
* ✨ Use pds resolver to get blob straight from pds
* ✅ Use FOR UPDATE to respect db transactions
* ♻️ Refactor to use event_pusher table instead of new table
* ✨ Bring back missing lines in pnpm-lock
* 🔨 Rebuild?
* 🚨 Formatting
* ♻️ Refactor to divert blob sync
* 🧹 Cleanup
* ✅ Use modClient seed client in blob-divert test
* update divert blob config to use basic admin auth
* fix
* build
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* sketching out label sequencer
* refactor sequencer
* sequencer tests
* tests
* add query labels endpoint & tests
* add pagination
* fix label formatting on temp
* tidy
* format labels
* make use listen/notify for sequencer
* ensure sig on all outgoing labels from ozone
* fixing up tests
* fix sequencer tests
* fix hanging server test
* add log on failure to update label
* update description for sig
* add expiration to labels
* fix test
* use bytes for label sigs
* fix tests
* add ver to labels
* tidy up background queue
* store signing keys as ids
* fix sequencer teest
* sketching out label sequencer
* refactor sequencer
* sequencer tests
* tests
* add query labels endpoint & tests
* add pagination
* fix label formatting on temp
* tidy
* format labels
* make use listen/notify for sequencer
* ensure sig on all outgoing labels from ozone
* fixing up tests
* fix sequencer tests
* fix hanging server test
* add log on failure to update label
* update description for sig
* use bytes for label sigs
* fix tests
* add ver to labels
* tidy up background queue
* store signing keys as ids
* fix sequencer teest
* adding some basic views
* feed hydration, add blocks to post hydration
* pass over notification hydration
* tidy
* merge
* implement getProfile
* hydrate post aggregation and viewer state
* fix
* fix codegen
* get some tests passing!
* add takedowns & some like bugfixing
* all profile tests passing!
* likes test
* follow endpoints using data plane
* reorg follow block rules
* reposts
* post views!
* implement getList w/ dataplane caveat
* adjust dataplane getListMembers to return listitem uris
* implement getListMutes and -Blocks w/ dataplane
* suggestions
* timeline
* misc view fixes
* view fixes for mutes, self-mute/block
* author feed
* feed gen routes
* tidy
* misc block/mute fixes
* list feed & actor likes
* implement getLists, fix some empty cursors
* implement getMutes, empty profile description fix
* implement getBlocks, block application fix
* implement getSuggestedFollowsByActor, needs some fixes
* feed generation
* search routes
* threads
* tidy
* fix some snaps
* fix getSuggestedFollowsByActor
* implement listNotifications
* implement getUnreadCount w/ dataplane
* implement notifications.updateSeen w/ dataplane
* 3rd party blocking tests
* blocked profile viewer
* add search mocks
* refactor getFeed
* createPipeline -> createPipelineNew
* basic replygating functionality on dataplane w/o filtering violating replies
* hack threadgates into dataplane, apply gates
* deterministic thread orders in dataplane
* misc cleanup around dataplane
* upgrade typescript to v5.3
* update typescript linter deps
* sync bsky proto, codegen
* update dataplane, sync with bsky proto updates
* remove indexer, ingester, daemon, moderation services from appview
* convert more bsky internals to dataplane, remove custom feedgens, implement mute/unmuting in mock dataplane
* remove bsky services. db and indexing logic into mock dataplane.
* remove tests not needed by appview v2, misc reorg
* add basic in-mem repo subscription to dataplane mock
* fix dev-env, bsky tests, bsky build
* cull bsky service entrypoint
* add bsky service readme
* build
* tidy
* tidy, fix pds proxy tests
* fix
* fix bsky entrypoint deps
* support http2 grpc client
* build
* fix dataplane bad tls config/default
* support multiple dataplane urls, retry when unavailable
* build
* tidy/fix
* move dataplane mock tests into their own dir
* cover label hydration through timeline test
* bring back labels in appview tests
* remove unused db primary/replica/coordinator from bsky dataplane
* bsky proto add cids to contracts, buf codegen
* sync-up bsky data-plane w/ codegen updates
* start using dataplane interaction endpoints
* add file
* avoid overfetching from dataplane, plumb feed items and cids
* pass refs through for post viewer state
* Lexicons: Add labeler prefs, labeler declaration, and get labeler routes
* Add labelerViewBasic and update embed views
* Fix typo
* switch list feeds to use feed item in dataplane
* handle not found err on get-thread dataplane call
* support use of search service rather than dataplane methods
* mark some appview v2 todos
* tidy
* still use dataplane on search endpoints when search service is not configured
* fix pds test
* Switch to labelerViewDetailed
* Move label and report values to refs
* Add getActorLabelers
* lint
* fix up bsky tests & snaps
* small diff to open pr
* rm new line
* codegen schemas
* tidy migrations
* table + indexing
* protos
* rename lexicons
* views, hydration + rename lexicons
* rest of routes
* data plane routes
* parse labelers from req
* fix appview-v2 docker build
* Support label issuer tied to appview v2 (#2033)
support label issuer tied to appview
* hydrate context
* tidy header logic
* integrating into more routes
* more routes
* wrap up rest
* add mock labeler
* rework labelerlexicons
* tidy lexs
* codegen new lexicons
* integrate lexicon rework
* add proxy logic
* forward labeler headers through pds
* tweak label header parsing
* remove did from scheams
* update indexing for lexs
* tests for mod service views
* label hydration test
* Add 'associated' to profileViewDetailed
* Rename labelers to mods in preferences
* Change uri to did in mod preferences
* couple more
* syntax tweaks
* integrate updated lexicons
* update view snap
* handle mod service embeds
* tidy
* fix mock
* lint
* base default labels of config var
* fix label hydration
* Appview v2: handle empty cursor on list notifications (#2017)
handle empty cursor on appview listnotifs
* Update appview v2 to use author feed enum (#2047)
* update bsky protos with author feed enum, misc feed item changes
* support new author feed enums in dataplane
* fix build
* Appview v2: utilize sorted-at field in bsky protos (#2050)
utilize new sorted-at field in bsky protos
* remove all dataplane usage of GetLikeCounts, switch to GetInteractionCounts
* Appview v2, sync w/ changes to protos (#2071)
* sync bsky protos
* sync-up bsky implementation w/ proto changes
* Appview v2 initial implementation for getPopularFeedGenerators (#2072)
add an initial implementation for getPopularFeedGenerators on appview v2
* merge
* fixes
* fix feed tests
* fix bsync mock
* format
* remove unused config
* fix lockfile
* another lockfile fix
* fix duplicate type
* fix dupplicate test
* Appview v2 handling clearly bad cursors (#2092)
* make mock dataplane cursors different from v1 cursors
* fail open on clearly bad appview cursors
* fix pds appview proxy snaps
* Appview v2 no notifs seen behavior (#2096)
* alter behavior for presenting notifications w/ no last-seen time
* fix pds proxy tests
* Appview v2 dataplane retries based on client host (#2098)
choose dataplane client for retries based on host when possible/relevant
* don't apply negated labels
* display suspensions on actor profile in appview v2
* Appview v2 use dataplane for identity lookups (#2095)
* update bsky proto w/ identity methods
* setup identity endpoints on mock dataplane
* move from idresolver to dataplane for identity lookups on appview
* tidy
* Appview v2: apply safe takedown refs to records, actors (#2107)
apply safe takedown refs to records, actors
* Fix timing on appview v2 repo rev header (#2113)
fix timing on appview repo rev
* fix post thread responses
* Appview v2 don't apply 3p self blocks (#2112)
do not apply 3p self-blocks
* Appview v2 search for feed generators (#2118)
* add protos for feedgen search
* support feed search on getPopularFeedGenerators
* Appview v2 config tidy (#2117)
* remove mod and triage roles from appview
* rename cdn and search config
* remove custom feed harness from appview v2
* Appview v2: don't apply missing modlists (#2122)
* dont apply missing mod lists
* update mock dataplane
* Update packages/bsky/src/hydration/hydrator.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* refactor & document a bit better
* fix up other routes
---------
Co-authored-by: devin ivy <devinivy@gmail.com>
* Appview v2 enforce post thread root boundary (#2120)
* enforce post thread root boundary
* test thread root boundary
* Appview v2 fix admin environment variable (#2137)
fix admin env in appview v2
* Remove re-pagination from getSuggestions (#2145)
* remove re-pagination from getSuggestions
* fix test
* Adjust wording for account suspension (#2153)
adjust wording for account suspension
* Appview v2: fix not-found and blocked uris in threads (#2201)
* fix uris of not-found and blocked posts in threads
* update snaps
* ✨ Show author feed of takendown author to admins only (#2197)
* fold in cid, auth, tracing, node version changes
* remove dead config from bsky service entrypoint
* build
* remove ozone test codepaths for appview v2
* tidy, docs fix
* fix test
* add additional user counts
* add associated data to profiles
* update snaps
* update to is_mod_service
* format
* tidy
* 3p labeler sdk updates (#2176)
* Update sdk to support 3p labeler preferences
* Stick with intolerance instead of hate for the label group id
* wip expand labels and label groups
* Output moderationOpts (computed) and modsPref (unaltered)
* Add tests for enabling/disabling mod services
* Add atproto-labelers header config
* Expand labels and label groups in definitions
* Fix tests
* Tweaks to labels
* Remove label descriptions and improve output types on labels and label groups
* Add mocker to exported API
* Improve types of label and label group definitions
* Rework moderation prefs to continue using global labelgroup settings and only disable label groups per moderator
* Simplify encoding of the label preferences in definitions
* Add target constraints to labels
* Refactor the moderation sdk to derive more behaviors from the definition files
* Small cleanup
* Add hiding tool to modsdk
* Track filter causes
* Make mute state an alert
* Fix: dont blur profileview for blocks
* Prioritize causes by severity
* Add moderateNotification() and drop quote post moderation code
* Add mocker functions for notifications
* Improve mock data
* Lexicon: Add custom label definitions and remove modservice descriptions
* Lexicon: Update moderation prefs
* SDK updates: remove label groups, reduce builtin labels, update mod-preference apis
* Lexicon: Update global labels with new reduced set
* Lexicon: Remove moderation.getService and add detailed option to getServices
* Lexicons: add severity=none to custom label value defs
* Implement custom label-value definition tooling
* All custom labels are no-self
* Backend impl for labeler lexicon updates (#2258)
* codegen
* clean up impl
* fix up tests
* Lexicon: modservice -> labeler
* Remove x- prefix behavior; add label value syntax rules; add custom label precedence rules
* Lexicon: Remove the ability to choose a defaultSetting from custom labels
* Rework test suites
* Give behaviors to all labels regardless of target
* sync up backend with lex changes
* fix labelers in dev-env agent
* lint protos
* update protos & views
* small bugfix & update tests
* tweak protos
* fix build issue from merge
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
* 🚧 Working through an nullable review state
* ✅ Update snapshots on some tests
* ✅ Update snapshots on some tests
* ✅ Add test for reviewOptional status mutation
* ♻️ Rename reviewOptional -> reviewNone
* ✅ Use FOR UPDATE to respect db transactions
* Allow tags to lead with and contain only numbers
* Break tags on other whitespace characters
* Export regexes from rich text detection
* Add test
* Add test
* Disallow number-only tags
* Avoid combining enclosing screen chars
* Allow full-width number sign
* Clarify tests
* Fix punctuation edge case
* Reorder
* Simplify, add another test
* Another test, comment
* Don't remove hash from muted words
* Split out crud actions, only sanitize on inserts
* Add changeset
* Handle hash emoji in mute words
* Add sanitization for invalid chars
* Remove console
* Add util
* Clean up changesets
* Format
* Wow forgot to commit change
* Version packages
* Give Mary credit
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eric Bailey <git@esb.lol>
* Do not parse hashtag emoji as tag (#2242)
* fix: prevent hashtag emoji from being parsed as tag
* chore: fmt
* fix: properly calculate length of tag
* Add a couple tests
---------
Co-authored-by: Mary <148872143+mary-ext@users.noreply.github.com>
* lex: tweak deprecation note
The motivation with this is that the docs site hides deprecated
endpoint, and the behavior was updated to only do this when
"deprecation" is the first word of the description (to avoid hiding some
endpoints which just talk about deprecation).
* make codegen
* draft of account migration lexicons
* format
* clean up schemas
* codegen
* second pass on schemas
* small fix
* move around checkImportStatus
* re-codegen
* getServiceAuth
* getServiceAuth impl
* importRepo impl
* handle uploadBlob for import
* allow bringing your own did on createAccount
* working on test flow
* fleshing out flow
* fix up sendPlcOP
* small fixes
* activate/deactivate account
* full flow working!
* schema tweaks
* format
* update schemas
* moar codegen
* match impl to new schemas
* email flow for signed plc operation
* add email flow for plc operations
* impl plc op request email
* proxy to entryway
* tidy activate account
* integrating account deactivated state
* fix up tests
* friendly parse on optional did auth
* admin activate/deactivate routes
* proxy relevant requests to entryway
* remove admin activation routes
* do not proxy acitvaition to entryway
* cfg for disallowing imports
* buff up test
* refactor listMissingBlobs a bit
* add validDid & activated to accoutn status
* emit event on account activation
* test creating a post after migrating
* account deactivation tests
* test name
* tests on plc operations
* fix recommended did creds
* codegen
* turn off accepting imports on createAccount
* undo prev change
* increment version
* build branch
* pr feedback
* handle errs in p-queue
* handle blob upload outside of txn
* Clean old temp account migration lexicons (#2187)
* clean old temp lexicons
* rm old test
* fix agent session test
* fix bsky test
* dont build branch
* draft of account migration lexicons
* format
* clean up schemas
* second pass on schemas
* small fix
* move around checkImportStatus
* getServiceAuth
* schema tweaks
* format
* update schemas
* email flow for signed plc operation
* refactor listMissingBlobs a bit
* codegen
* return did in describeServer
* changeset
* ✨ Store languages on subjects from record data
* ✨ Fetch lang from author feed for repo subject
* ✨ Regenerate lex
* ✨ Add lang to profile subject
* ✨ Move lang to flag
* ✨ Add comment to flag event
* ✅ Update pds test snapshot
* 🐛 Fix broken import
* ✨ Rename flag to tag
* ✅ Update snapshot for event by id
* ✅ Bring back skipped test
* ✨ Move tags to dedicated columns
* ✅ Update test snapshots, use logger
* ✨ Change lang:unknown to lang:und
* ✅ Update test snapshots
* 🧹 Cleanup
* ✨ Add filter params for tags on queryModerationEvents endpoint
* ✨ Add tags property to subject status model
* ✅ Update test code
* ✨ Add filter to get subjects that do not have certain tags
* ✨ Add initial lexicons to manage communication templates
* ✨ All 3 endpoints are functional
* ✨ Add list and delete endpoints
* ✅ Add tests for communication template CRUD
* 🔒 Allow only admins to create and update templates
* 🧹 Cleanup according to PR review
* ✨ Make updatedBy and createdBy optional in lexicon
* ✨ Typo
* ✨ Allow string id and update lexicon language
* ✅ Fix tests
* ✨ content -> contentMarkdown
* ✨ Change column name in db table
* add changeset
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* mv appview
* copy
* finalize copy
* package names
* big WIP
* first pass at mod servce
* some tidy
* tidy & fix compiler errors
* rename to ozone, db migrations, add to dev-env & pds cfg
* getRecord & getRepo mostly working
* fix open handle
* get record tests all working
* moderation events working
* statuses working
* tidy test suite
* search repos
* server & db tests
* moderation tests
* wip daemon + push events
* pds fanout working
* fix db test
* fanning takedowns out to appview
* rm try/catch
* bsky moderation test
* introduce mod subject wrappers
* more tidy
* refactor event reversal
* tidy some db stuff
* tidy
* rename service to mod-service
* fix test
* tidy config
* refactor auth in bsky
* wip patching up auto-mod
* add label ingester in appview
* fix a couple build issues
* fix some timing bugs
* tidy polling logic
* fix up tests
* fix some pds tests
* eslint ignore
* fix ozone tests
* move seeds to dev-env
* move images around
* fix db schemas
* use service auth admin reqs
* fix remaining tests
* auth tests bsky
* another test
* random tidy
* fix up search
* clean up bsky mod service
* more tidy
* default attempts to 0
* tidy old test
* random tidy
* tidy package.json
* tidy logger
* takedownId -> takedownRef
* misc pr feedback
* split daemon out from ozone application
* fix blob takedown mgiration
* refactor ozone config
* do push event fanout on write instead of on read
* make suspend error work again
* add attempts check & add supporting index
* fix takedown test ref
* get tests working
* rm old test
* fix timing bug in event pusher tests
* attempt another fix for timing bug
* await req
* service files
* remove labelerDid cfg
* update snaps for labeler did + some cfg changes
* fix more snaps
* pnpm i
* build ozone images
* build
* make label provider optional
* fix build issues
* fix build
* fix build
* build pds
* build on ghcr
* fix syntax in entry
* another fix
* use correct import
* export logger
* remove event reverser
* adjust push event fanout
* push out multiple
* remove builds
