* remove `neg` on the labels that ozone emits
* no negs in appview either
* no negs on self-labels
* no neg on self labels in ozone either
* fix query labels test
* Rename bsky_labeler_did
* Use labeldef default setting
* Add definedBy to interpretted label defs
* Improve dev-env mocks for labels
* Remove global label defs for dmca-violation, doxxing, and !no-promote
* Change nudity global label def to default to ignore and no longer be adult-only
* Remove old !no-promote tests
* Add mod authorities header and move bsky labeler into it
* Rename modsPref and addModService/removeModService to labelersPref and add/removeLabeler
* Add defaultSetting and adultOnly to custom label value definitions
* Rename InterprettedLabelValueDefinition to InterpretedLabelValueDefinition
* Update dev-env mock
* Move muted words and hidden posts into moderationPrefs
* Add muted word and hidden post handling to moderatePost
* Add mutewords tests
* Finish muteword tests
* Add mod-authority.test to dev-env
* Rename global label value def gore to graphic-media
* Fix typo
* Stop converting old label values
* Update label target=profile behaviors: dont filter content on hide, dont blur display names, but do show the info cards
* Update label target=account behaviors: dont blur media of content
* Add muteword moderation behaviors
* Fix mock label defs
* Implement quote-post moderation handling
* Add adult content test
* Handle sync legacy labels (#2291)
* Handle sync legacy labels
* Remap values on read
* Filter out double-written legacy label values
* Better naming, fix types
* Fix test
* Update moderation docs in sdk
* Update to new atproto-accept-labelers header behavior
* Add getLabelDefinitions() helper method
* Add proxy header support to agent
* Update mock moderation
* lint
* Implement moderation for userlists and feedgens
* Add another test label
* fix labeler in dev-env agents
* fix label hydration test
* fix lint error
* fix agent test
* fix takedown labels test
---------
Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: dholms <dtholmgren@gmail.com>
* lexicons
* clean up old admin schemas
* codegen
* renaming all routes
* fix more
* fix snaps
* fix test
* format
* hook up ozone routes on pds
* publish next version
* build branch
* dont build branch
* lexicon: add labels to feedgen and list views
* appview: hydrate labels onto lists, feedgens
* test hydration of labels on lists and feedgens
* update pds snap
* 🚧 Working through an nullable review state
* ✅ Update snapshots on some tests
* ✅ Update snapshots on some tests
* ✅ Add test for reviewOptional status mutation
* ✨ Add divertBlobs event to send blobs to abyss
* ♻️ Rename reviewOptional -> reviewNone
* ♻️ Rename modEventDivertBlobs -> modEventDivert
* 🐛 Rename event type checker
* ✨ Use pds resolver to get blob straight from pds
* ✅ Use FOR UPDATE to respect db transactions
* ♻️ Refactor to use event_pusher table instead of new table
* ✨ Bring back missing lines in pnpm-lock
* 🔨 Rebuild?
* 🚨 Formatting
* ♻️ Refactor to divert blob sync
* 🧹 Cleanup
* ✅ Use modClient seed client in blob-divert test
* update divert blob config to use basic admin auth
* fix
* build
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* sketching out label sequencer
* refactor sequencer
* sequencer tests
* tests
* add query labels endpoint & tests
* add pagination
* fix label formatting on temp
* tidy
* format labels
* make use listen/notify for sequencer
* ensure sig on all outgoing labels from ozone
* fixing up tests
* fix sequencer tests
* fix hanging server test
* add log on failure to update label
* update description for sig
* add expiration to labels
* fix test
* use bytes for label sigs
* fix tests
* add ver to labels
* tidy up background queue
* store signing keys as ids
* fix sequencer teest
* sketching out label sequencer
* refactor sequencer
* sequencer tests
* tests
* add query labels endpoint & tests
* add pagination
* fix label formatting on temp
* tidy
* format labels
* make use listen/notify for sequencer
* ensure sig on all outgoing labels from ozone
* fixing up tests
* fix sequencer tests
* fix hanging server test
* add log on failure to update label
* update description for sig
* use bytes for label sigs
* fix tests
* add ver to labels
* tidy up background queue
* store signing keys as ids
* fix sequencer teest
* tidy bsky auth
* hook up new auth verifier
* update auth throughout ozone
* handle mod signing keys
* add client proxy heads to pds
* hook up rest of routes
* simplify pipethrough & add some SSRF protection
* tests
* fix bad var
* remove basic auth in ozone
* wip
* fix key parsing in pds
* fix up all ozone tests
* fix admin auth test
* rename test
* fix ozone test
* clean up tokens in pds
* fix up pds tests
* fix up ozone tests
* add pipethrough to write routes
* reenable proxied admin test
* add moderator accounts to ozone in dev-env
* update did doc id values
* null creds string -> `none`
* fix fetchLabels auth check
* ✨ Add a couple more proxied requests that we use in ozone ui
* Add runit to the services/bsky Dockerfile (#2254)
add runit to the services/bsky Dockerfile
* Improve tag detection (#2260)
* Allow tags to lead with and contain only numbers
* Break tags on other whitespace characters
* Export regexes from rich text detection
* Add test
* Add test
* Disallow number-only tags
* Avoid combining enclosing screen chars
* Allow full-width number sign
* Clarify tests
* Fix punctuation edge case
* Reorder
* Simplify, add another test
* Another test, comment
* Version packages (#2261)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* 🐛 Increment attempt count after each attempt to push ozone event (#2239)
* Ozone delegates email sending to actor's pds (#2272)
* ozone delegates email sending to user's pds
* lexicon: add content field to mod email event
* test email sending via mod event
* add dev dep for nodemailer in ozone
* fix auth verifier method
* build branch
* build branch
* fix url check
* better error handling for get account infos
* fix labeler service id
* fix iss on auth headers
* fix dev-env ozone did
* fix tests & another jwt issuer
* fix proxy auth
* ozone: fix ip check
* fix aud check on pds mod service auth
* tidy
* Update packages/pds/tests/proxied/admin.test.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* fix pipethrough of headers
* fix moderation status tests
* fix auth on ozone routes
* update iss on daemon
---------
Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
Co-authored-by: Jake Gold <52801504+Jacob2161@users.noreply.github.com>
Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: devin ivy <devinivy@gmail.com>
* adding some basic views
* feed hydration, add blocks to post hydration
* pass over notification hydration
* tidy
* merge
* implement getProfile
* hydrate post aggregation and viewer state
* fix
* fix codegen
* get some tests passing!
* add takedowns & some like bugfixing
* all profile tests passing!
* likes test
* follow endpoints using data plane
* reorg follow block rules
* reposts
* post views!
* implement getList w/ dataplane caveat
* adjust dataplane getListMembers to return listitem uris
* implement getListMutes and -Blocks w/ dataplane
* suggestions
* timeline
* misc view fixes
* view fixes for mutes, self-mute/block
* author feed
* feed gen routes
* tidy
* misc block/mute fixes
* list feed & actor likes
* implement getLists, fix some empty cursors
* implement getMutes, empty profile description fix
* implement getBlocks, block application fix
* implement getSuggestedFollowsByActor, needs some fixes
* feed generation
* search routes
* threads
* tidy
* fix some snaps
* fix getSuggestedFollowsByActor
* implement listNotifications
* implement getUnreadCount w/ dataplane
* implement notifications.updateSeen w/ dataplane
* 3rd party blocking tests
* blocked profile viewer
* add search mocks
* refactor getFeed
* createPipeline -> createPipelineNew
* basic replygating functionality on dataplane w/o filtering violating replies
* hack threadgates into dataplane, apply gates
* deterministic thread orders in dataplane
* misc cleanup around dataplane
* upgrade typescript to v5.3
* update typescript linter deps
* sync bsky proto, codegen
* update dataplane, sync with bsky proto updates
* remove indexer, ingester, daemon, moderation services from appview
* convert more bsky internals to dataplane, remove custom feedgens, implement mute/unmuting in mock dataplane
* remove bsky services. db and indexing logic into mock dataplane.
* remove tests not needed by appview v2, misc reorg
* add basic in-mem repo subscription to dataplane mock
* fix dev-env, bsky tests, bsky build
* cull bsky service entrypoint
* add bsky service readme
* build
* tidy
* tidy, fix pds proxy tests
* fix
* fix bsky entrypoint deps
* support http2 grpc client
* build
* fix dataplane bad tls config/default
* support multiple dataplane urls, retry when unavailable
* build
* tidy/fix
* move dataplane mock tests into their own dir
* cover label hydration through timeline test
* bring back labels in appview tests
* remove unused db primary/replica/coordinator from bsky dataplane
* bsky proto add cids to contracts, buf codegen
* sync-up bsky data-plane w/ codegen updates
* start using dataplane interaction endpoints
* add file
* avoid overfetching from dataplane, plumb feed items and cids
* pass refs through for post viewer state
* Lexicons: Add labeler prefs, labeler declaration, and get labeler routes
* Add labelerViewBasic and update embed views
* Fix typo
* switch list feeds to use feed item in dataplane
* handle not found err on get-thread dataplane call
* support use of search service rather than dataplane methods
* mark some appview v2 todos
* tidy
* still use dataplane on search endpoints when search service is not configured
* fix pds test
* Switch to labelerViewDetailed
* Move label and report values to refs
* Add getActorLabelers
* lint
* fix up bsky tests & snaps
* small diff to open pr
* rm new line
* codegen schemas
* tidy migrations
* table + indexing
* protos
* rename lexicons
* views, hydration + rename lexicons
* rest of routes
* data plane routes
* parse labelers from req
* fix appview-v2 docker build
* Support label issuer tied to appview v2 (#2033)
support label issuer tied to appview
* hydrate context
* tidy header logic
* integrating into more routes
* more routes
* wrap up rest
* add mock labeler
* rework labelerlexicons
* tidy lexs
* codegen new lexicons
* integrate lexicon rework
* add proxy logic
* forward labeler headers through pds
* tweak label header parsing
* remove did from scheams
* update indexing for lexs
* tests for mod service views
* label hydration test
* Add 'associated' to profileViewDetailed
* Rename labelers to mods in preferences
* Change uri to did in mod preferences
* couple more
* syntax tweaks
* integrate updated lexicons
* update view snap
* handle mod service embeds
* tidy
* fix mock
* lint
* base default labels of config var
* fix label hydration
* Appview v2: handle empty cursor on list notifications (#2017)
handle empty cursor on appview listnotifs
* Update appview v2 to use author feed enum (#2047)
* update bsky protos with author feed enum, misc feed item changes
* support new author feed enums in dataplane
* fix build
* Appview v2: utilize sorted-at field in bsky protos (#2050)
utilize new sorted-at field in bsky protos
* remove all dataplane usage of GetLikeCounts, switch to GetInteractionCounts
* Appview v2, sync w/ changes to protos (#2071)
* sync bsky protos
* sync-up bsky implementation w/ proto changes
* Appview v2 initial implementation for getPopularFeedGenerators (#2072)
add an initial implementation for getPopularFeedGenerators on appview v2
* merge
* fixes
* fix feed tests
* fix bsync mock
* format
* remove unused config
* fix lockfile
* another lockfile fix
* fix duplicate type
* fix dupplicate test
* Appview v2 handling clearly bad cursors (#2092)
* make mock dataplane cursors different from v1 cursors
* fail open on clearly bad appview cursors
* fix pds appview proxy snaps
* Appview v2 no notifs seen behavior (#2096)
* alter behavior for presenting notifications w/ no last-seen time
* fix pds proxy tests
* Appview v2 dataplane retries based on client host (#2098)
choose dataplane client for retries based on host when possible/relevant
* don't apply negated labels
* display suspensions on actor profile in appview v2
* Appview v2 use dataplane for identity lookups (#2095)
* update bsky proto w/ identity methods
* setup identity endpoints on mock dataplane
* move from idresolver to dataplane for identity lookups on appview
* tidy
* Appview v2: apply safe takedown refs to records, actors (#2107)
apply safe takedown refs to records, actors
* Fix timing on appview v2 repo rev header (#2113)
fix timing on appview repo rev
* fix post thread responses
* Appview v2 don't apply 3p self blocks (#2112)
do not apply 3p self-blocks
* Appview v2 search for feed generators (#2118)
* add protos for feedgen search
* support feed search on getPopularFeedGenerators
* Appview v2 config tidy (#2117)
* remove mod and triage roles from appview
* rename cdn and search config
* remove custom feed harness from appview v2
* Appview v2: don't apply missing modlists (#2122)
* dont apply missing mod lists
* update mock dataplane
* Update packages/bsky/src/hydration/hydrator.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* refactor & document a bit better
* fix up other routes
---------
Co-authored-by: devin ivy <devinivy@gmail.com>
* Appview v2 enforce post thread root boundary (#2120)
* enforce post thread root boundary
* test thread root boundary
* Appview v2 fix admin environment variable (#2137)
fix admin env in appview v2
* Remove re-pagination from getSuggestions (#2145)
* remove re-pagination from getSuggestions
* fix test
* Adjust wording for account suspension (#2153)
adjust wording for account suspension
* Appview v2: fix not-found and blocked uris in threads (#2201)
* fix uris of not-found and blocked posts in threads
* update snaps
* ✨ Show author feed of takendown author to admins only (#2197)
* fold in cid, auth, tracing, node version changes
* remove dead config from bsky service entrypoint
* build
* remove ozone test codepaths for appview v2
* tidy, docs fix
* fix test
* add additional user counts
* add associated data to profiles
* update snaps
* update to is_mod_service
* format
* tidy
* 3p labeler sdk updates (#2176)
* Update sdk to support 3p labeler preferences
* Stick with intolerance instead of hate for the label group id
* wip expand labels and label groups
* Output moderationOpts (computed) and modsPref (unaltered)
* Add tests for enabling/disabling mod services
* Add atproto-labelers header config
* Expand labels and label groups in definitions
* Fix tests
* Tweaks to labels
* Remove label descriptions and improve output types on labels and label groups
* Add mocker to exported API
* Improve types of label and label group definitions
* Rework moderation prefs to continue using global labelgroup settings and only disable label groups per moderator
* Simplify encoding of the label preferences in definitions
* Add target constraints to labels
* Refactor the moderation sdk to derive more behaviors from the definition files
* Small cleanup
* Add hiding tool to modsdk
* Track filter causes
* Make mute state an alert
* Fix: dont blur profileview for blocks
* Prioritize causes by severity
* Add moderateNotification() and drop quote post moderation code
* Add mocker functions for notifications
* Improve mock data
* Lexicon: Add custom label definitions and remove modservice descriptions
* Lexicon: Update moderation prefs
* SDK updates: remove label groups, reduce builtin labels, update mod-preference apis
* Lexicon: Update global labels with new reduced set
* Lexicon: Remove moderation.getService and add detailed option to getServices
* Lexicons: add severity=none to custom label value defs
* Implement custom label-value definition tooling
* All custom labels are no-self
* Backend impl for labeler lexicon updates (#2258)
* codegen
* clean up impl
* fix up tests
* Lexicon: modservice -> labeler
* Remove x- prefix behavior; add label value syntax rules; add custom label precedence rules
* Lexicon: Remove the ability to choose a defaultSetting from custom labels
* Rework test suites
* Give behaviors to all labels regardless of target
* sync up backend with lex changes
* fix labelers in dev-env agent
* lint protos
* update protos & views
* small bugfix & update tests
* tweak protos
* fix build issue from merge
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
* 🚧 Working through an nullable review state
* ✅ Update snapshots on some tests
* ✅ Update snapshots on some tests
* ✅ Add test for reviewOptional status mutation
* ♻️ Rename reviewOptional -> reviewNone
* ✅ Use FOR UPDATE to respect db transactions
* tidy bsky auth
* hook up new auth verifier
* update auth throughout ozone
* handle mod signing keys
* fix bad var
* fix key parsing in pds
* fix admin auth test
* rename test
* update did doc id values
* null creds string -> `none`
* fix fetchLabels auth check
* ✨ Add a couple more proxied requests that we use in ozone ui
* Add runit to the services/bsky Dockerfile (#2254)
add runit to the services/bsky Dockerfile
* Improve tag detection (#2260)
* Allow tags to lead with and contain only numbers
* Break tags on other whitespace characters
* Export regexes from rich text detection
* Add test
* Add test
* Disallow number-only tags
* Avoid combining enclosing screen chars
* Allow full-width number sign
* Clarify tests
* Fix punctuation edge case
* Reorder
* Simplify, add another test
* Another test, comment
* Version packages (#2261)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* 🐛 Increment attempt count after each attempt to push ozone event (#2239)
* Ozone delegates email sending to actor's pds (#2272)
* ozone delegates email sending to user's pds
* lexicon: add content field to mod email event
* test email sending via mod event
* fix auth verifier method
* better error handling for get account infos
* fix labeler service id
* fix iss on auth headers
* fix dev-env ozone did
* fix tests & another jwt issuer
* ozone: fix ip check
* fix aud check on pds mod service auth
* tidy
---------
Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
Co-authored-by: Jake Gold <52801504+Jacob2161@users.noreply.github.com>
Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: devin ivy <devinivy@gmail.com>
* hooking up invalidator to ozone
* test + fix
* wire up service entry
* add cid to invalidation url
* add aws to ozone service package.json
* build branch
* fix build
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* Version packages
* Give Mary credit
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eric Bailey <git@esb.lol>
* lex: tweak deprecation note
The motivation with this is that the docs site hides deprecated
endpoint, and the behavior was updated to only do this when
"deprecation" is the first word of the description (to avoid hiding some
endpoints which just talk about deprecation).
* make codegen
* draft of account migration lexicons
* format
* clean up schemas
* codegen
* second pass on schemas
* small fix
* move around checkImportStatus
* re-codegen
* getServiceAuth
* getServiceAuth impl
* importRepo impl
* handle uploadBlob for import
* allow bringing your own did on createAccount
* working on test flow
* fleshing out flow
* fix up sendPlcOP
* small fixes
* activate/deactivate account
* full flow working!
* schema tweaks
* format
* update schemas
* moar codegen
* match impl to new schemas
* email flow for signed plc operation
* add email flow for plc operations
* impl plc op request email
* proxy to entryway
* tidy activate account
* integrating account deactivated state
* fix up tests
* friendly parse on optional did auth
* admin activate/deactivate routes
* proxy relevant requests to entryway
* remove admin activation routes
* do not proxy acitvaition to entryway
* cfg for disallowing imports
* buff up test
* refactor listMissingBlobs a bit
* add validDid & activated to accoutn status
* emit event on account activation
* test creating a post after migrating
* account deactivation tests
* test name
* tests on plc operations
* fix recommended did creds
* codegen
* turn off accepting imports on createAccount
* undo prev change
* increment version
* build branch
* pr feedback
* handle errs in p-queue
* handle blob upload outside of txn
* Clean old temp account migration lexicons (#2187)
* clean old temp lexicons
* rm old test
* fix agent session test
* fix bsky test
* dont build branch
* draft of account migration lexicons
* format
* clean up schemas
* second pass on schemas
* small fix
* move around checkImportStatus
* getServiceAuth
* schema tweaks
* format
* update schemas
* email flow for signed plc operation
* refactor listMissingBlobs a bit
* codegen
* return did in describeServer
* changeset
* ✨ Store languages on subjects from record data
* ✨ Fetch lang from author feed for repo subject
* ✨ Regenerate lex
* ✨ Add lang to profile subject
* ✨ Move lang to flag
* ✨ Add comment to flag event
* ✅ Update pds test snapshot
* 🐛 Fix broken import
* ✨ Rename flag to tag
* ✅ Update snapshot for event by id
* ✅ Bring back skipped test
* ✨ Move tags to dedicated columns
* ✅ Update test snapshots, use logger
* ✨ Change lang:unknown to lang:und
* ✅ Update test snapshots
* 🧹 Cleanup
* ✨ Add filter params for tags on queryModerationEvents endpoint
* ✨ Add tags property to subject status model
* ✅ Update test code
* ✨ Add filter to get subjects that do not have certain tags
