* feat(api): support creation of oauth based AtpAgents
* oauth: misc fixes for confidential clients
* fix(xprc): remove ReadableStream.from polyfill
* OAuth docs tweaks (#2679)
* OAuth: clarification about client_name being shown
* OAuth: re-write handle resolution privacy concern
* avoid relying on ReadableStream.from in xrpc-server tests
* feat(oauth-types): expose "ALLOW_UNSECURE_ORIGINS" constant
* feat(handle-resolver): expose "AtprotoIdentityDidMethods" type
* fix(oauth-client): ensure that the oauth metadata document contains client_id_metadata_document_supported
* fix(oauth-types): prevent unknown query string in loopback client id
* fix(identity-resolver): check that handle is in did doc's "alsoKnownAs"
* feat(oauth-client:oauth-resolver): allow logging in using either the PDS URL or Entryway URL
* fix(oauth-client): return better error in case of invalid "oauth-protected-resource" status code
* refactor(did): group atproto specific checks in own
* feat(api): relax typing of "appLabelers" and "labelers" AtpClient properties
* allow any did as labeller (for tests mainly)
* fix(api): allow to override "atproto-proxy" on a per-request basis
* remove release candidate versions from changelog
* update changeset for api and xrpc packages
* Add missing changeset
* revert RC versions
* Proper wording in OAUTH.md api example
* remove "pre" changeset file
* xrpc: restore original behavior of setHEader and unsetHeader
* docs: add comment for XrpcClient 's constructor arg
* feat(api): expose "schemas" publicly
* feat(api): allow customizing the whatwg fetch function of the AtpAgent
* docs(api): improve migration docs
* docs: change reference to BskyAgent to AtpAgent
* docs: mention the breaking change regarding setSessionPersistHandler
* fix(api): better split AtpClient concerns
* fix(xrpc): remove unused import
* refactor(api): simplify class hierarchu by removeing AtpClient
* fix(api): mock proper method for facets detection
* restore ability to restore session asynchronously
* feat(api): allow instantiating Agent with same argument as super class
* docs(api): properly extend Agent class
* style(xrpc): var name
* docs(api): remove "async" to header getter
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Hailey <me@haileyok.com>
* Rename bsky_labeler_did
* Use labeldef default setting
* Add definedBy to interpretted label defs
* Improve dev-env mocks for labels
* Remove global label defs for dmca-violation, doxxing, and !no-promote
* Change nudity global label def to default to ignore and no longer be adult-only
* Remove old !no-promote tests
* Add mod authorities header and move bsky labeler into it
* Rename modsPref and addModService/removeModService to labelersPref and add/removeLabeler
* Add defaultSetting and adultOnly to custom label value definitions
* Rename InterprettedLabelValueDefinition to InterpretedLabelValueDefinition
* Update dev-env mock
* Move muted words and hidden posts into moderationPrefs
* Add muted word and hidden post handling to moderatePost
* Add mutewords tests
* Finish muteword tests
* Add mod-authority.test to dev-env
* Rename global label value def gore to graphic-media
* Fix typo
* Stop converting old label values
* Update label target=profile behaviors: dont filter content on hide, dont blur display names, but do show the info cards
* Update label target=account behaviors: dont blur media of content
* Add muteword moderation behaviors
* Fix mock label defs
* Implement quote-post moderation handling
* Add adult content test
* Handle sync legacy labels (#2291)
* Handle sync legacy labels
* Remap values on read
* Filter out double-written legacy label values
* Better naming, fix types
* Fix test
* Update moderation docs in sdk
* Update to new atproto-accept-labelers header behavior
* Add getLabelDefinitions() helper method
* Add proxy header support to agent
* Update mock moderation
* lint
* Implement moderation for userlists and feedgens
* Add another test label
* fix labeler in dev-env agents
* fix label hydration test
* fix lint error
* fix agent test
* fix takedown labels test
---------
Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: dholms <dtholmgren@gmail.com>
* adding some basic views
* feed hydration, add blocks to post hydration
* pass over notification hydration
* tidy
* merge
* implement getProfile
* hydrate post aggregation and viewer state
* fix
* fix codegen
* get some tests passing!
* add takedowns & some like bugfixing
* all profile tests passing!
* likes test
* follow endpoints using data plane
* reorg follow block rules
* reposts
* post views!
* implement getList w/ dataplane caveat
* adjust dataplane getListMembers to return listitem uris
* implement getListMutes and -Blocks w/ dataplane
* suggestions
* timeline
* misc view fixes
* view fixes for mutes, self-mute/block
* author feed
* feed gen routes
* tidy
* misc block/mute fixes
* list feed & actor likes
* implement getLists, fix some empty cursors
* implement getMutes, empty profile description fix
* implement getBlocks, block application fix
* implement getSuggestedFollowsByActor, needs some fixes
* feed generation
* search routes
* threads
* tidy
* fix some snaps
* fix getSuggestedFollowsByActor
* implement listNotifications
* implement getUnreadCount w/ dataplane
* implement notifications.updateSeen w/ dataplane
* 3rd party blocking tests
* blocked profile viewer
* add search mocks
* refactor getFeed
* createPipeline -> createPipelineNew
* basic replygating functionality on dataplane w/o filtering violating replies
* hack threadgates into dataplane, apply gates
* deterministic thread orders in dataplane
* misc cleanup around dataplane
* upgrade typescript to v5.3
* update typescript linter deps
* sync bsky proto, codegen
* update dataplane, sync with bsky proto updates
* remove indexer, ingester, daemon, moderation services from appview
* convert more bsky internals to dataplane, remove custom feedgens, implement mute/unmuting in mock dataplane
* remove bsky services. db and indexing logic into mock dataplane.
* remove tests not needed by appview v2, misc reorg
* add basic in-mem repo subscription to dataplane mock
* fix dev-env, bsky tests, bsky build
* cull bsky service entrypoint
* add bsky service readme
* build
* tidy
* tidy, fix pds proxy tests
* fix
* fix bsky entrypoint deps
* support http2 grpc client
* build
* fix dataplane bad tls config/default
* support multiple dataplane urls, retry when unavailable
* build
* tidy/fix
* move dataplane mock tests into their own dir
* cover label hydration through timeline test
* bring back labels in appview tests
* remove unused db primary/replica/coordinator from bsky dataplane
* bsky proto add cids to contracts, buf codegen
* sync-up bsky data-plane w/ codegen updates
* start using dataplane interaction endpoints
* add file
* avoid overfetching from dataplane, plumb feed items and cids
* pass refs through for post viewer state
* Lexicons: Add labeler prefs, labeler declaration, and get labeler routes
* Add labelerViewBasic and update embed views
* Fix typo
* switch list feeds to use feed item in dataplane
* handle not found err on get-thread dataplane call
* support use of search service rather than dataplane methods
* mark some appview v2 todos
* tidy
* still use dataplane on search endpoints when search service is not configured
* fix pds test
* Switch to labelerViewDetailed
* Move label and report values to refs
* Add getActorLabelers
* lint
* fix up bsky tests & snaps
* small diff to open pr
* rm new line
* codegen schemas
* tidy migrations
* table + indexing
* protos
* rename lexicons
* views, hydration + rename lexicons
* rest of routes
* data plane routes
* parse labelers from req
* fix appview-v2 docker build
* Support label issuer tied to appview v2 (#2033)
support label issuer tied to appview
* hydrate context
* tidy header logic
* integrating into more routes
* more routes
* wrap up rest
* add mock labeler
* rework labelerlexicons
* tidy lexs
* codegen new lexicons
* integrate lexicon rework
* add proxy logic
* forward labeler headers through pds
* tweak label header parsing
* remove did from scheams
* update indexing for lexs
* tests for mod service views
* label hydration test
* Add 'associated' to profileViewDetailed
* Rename labelers to mods in preferences
* Change uri to did in mod preferences
* couple more
* syntax tweaks
* integrate updated lexicons
* update view snap
* handle mod service embeds
* tidy
* fix mock
* lint
* base default labels of config var
* fix label hydration
* Appview v2: handle empty cursor on list notifications (#2017)
handle empty cursor on appview listnotifs
* Update appview v2 to use author feed enum (#2047)
* update bsky protos with author feed enum, misc feed item changes
* support new author feed enums in dataplane
* fix build
* Appview v2: utilize sorted-at field in bsky protos (#2050)
utilize new sorted-at field in bsky protos
* remove all dataplane usage of GetLikeCounts, switch to GetInteractionCounts
* Appview v2, sync w/ changes to protos (#2071)
* sync bsky protos
* sync-up bsky implementation w/ proto changes
* Appview v2 initial implementation for getPopularFeedGenerators (#2072)
add an initial implementation for getPopularFeedGenerators on appview v2
* merge
* fixes
* fix feed tests
* fix bsync mock
* format
* remove unused config
* fix lockfile
* another lockfile fix
* fix duplicate type
* fix dupplicate test
* Appview v2 handling clearly bad cursors (#2092)
* make mock dataplane cursors different from v1 cursors
* fail open on clearly bad appview cursors
* fix pds appview proxy snaps
* Appview v2 no notifs seen behavior (#2096)
* alter behavior for presenting notifications w/ no last-seen time
* fix pds proxy tests
* Appview v2 dataplane retries based on client host (#2098)
choose dataplane client for retries based on host when possible/relevant
* don't apply negated labels
* display suspensions on actor profile in appview v2
* Appview v2 use dataplane for identity lookups (#2095)
* update bsky proto w/ identity methods
* setup identity endpoints on mock dataplane
* move from idresolver to dataplane for identity lookups on appview
* tidy
* Appview v2: apply safe takedown refs to records, actors (#2107)
apply safe takedown refs to records, actors
* Fix timing on appview v2 repo rev header (#2113)
fix timing on appview repo rev
* fix post thread responses
* Appview v2 don't apply 3p self blocks (#2112)
do not apply 3p self-blocks
* Appview v2 search for feed generators (#2118)
* add protos for feedgen search
* support feed search on getPopularFeedGenerators
* Appview v2 config tidy (#2117)
* remove mod and triage roles from appview
* rename cdn and search config
* remove custom feed harness from appview v2
* Appview v2: don't apply missing modlists (#2122)
* dont apply missing mod lists
* update mock dataplane
* Update packages/bsky/src/hydration/hydrator.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* refactor & document a bit better
* fix up other routes
---------
Co-authored-by: devin ivy <devinivy@gmail.com>
* Appview v2 enforce post thread root boundary (#2120)
* enforce post thread root boundary
* test thread root boundary
* Appview v2 fix admin environment variable (#2137)
fix admin env in appview v2
* Remove re-pagination from getSuggestions (#2145)
* remove re-pagination from getSuggestions
* fix test
* Adjust wording for account suspension (#2153)
adjust wording for account suspension
* Appview v2: fix not-found and blocked uris in threads (#2201)
* fix uris of not-found and blocked posts in threads
* update snaps
* ✨ Show author feed of takendown author to admins only (#2197)
* fold in cid, auth, tracing, node version changes
* remove dead config from bsky service entrypoint
* build
* remove ozone test codepaths for appview v2
* tidy, docs fix
* fix test
* add additional user counts
* add associated data to profiles
* update snaps
* update to is_mod_service
* format
* tidy
* 3p labeler sdk updates (#2176)
* Update sdk to support 3p labeler preferences
* Stick with intolerance instead of hate for the label group id
* wip expand labels and label groups
* Output moderationOpts (computed) and modsPref (unaltered)
* Add tests for enabling/disabling mod services
* Add atproto-labelers header config
* Expand labels and label groups in definitions
* Fix tests
* Tweaks to labels
* Remove label descriptions and improve output types on labels and label groups
* Add mocker to exported API
* Improve types of label and label group definitions
* Rework moderation prefs to continue using global labelgroup settings and only disable label groups per moderator
* Simplify encoding of the label preferences in definitions
* Add target constraints to labels
* Refactor the moderation sdk to derive more behaviors from the definition files
* Small cleanup
* Add hiding tool to modsdk
* Track filter causes
* Make mute state an alert
* Fix: dont blur profileview for blocks
* Prioritize causes by severity
* Add moderateNotification() and drop quote post moderation code
* Add mocker functions for notifications
* Improve mock data
* Lexicon: Add custom label definitions and remove modservice descriptions
* Lexicon: Update moderation prefs
* SDK updates: remove label groups, reduce builtin labels, update mod-preference apis
* Lexicon: Update global labels with new reduced set
* Lexicon: Remove moderation.getService and add detailed option to getServices
* Lexicons: add severity=none to custom label value defs
* Implement custom label-value definition tooling
* All custom labels are no-self
* Backend impl for labeler lexicon updates (#2258)
* codegen
* clean up impl
* fix up tests
* Lexicon: modservice -> labeler
* Remove x- prefix behavior; add label value syntax rules; add custom label precedence rules
* Lexicon: Remove the ability to choose a defaultSetting from custom labels
* Rework test suites
* Give behaviors to all labels regardless of target
* sync up backend with lex changes
* fix labelers in dev-env agent
* lint protos
* update protos & views
* small bugfix & update tests
* tweak protos
* fix build issue from merge
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
* Add self-label schemas
* switch around array and union for self-labels
* apply self-labels to post and profile views
* test for self labels in pds
* apply self-labels to post and profile views in bsky appview
* test for self labels in bsky appview
* update pds proxied test snapshots
* Add support for self-labels to the mdoeration sdk
* Disable unknown-labeler filtering until 3P support is added
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* First pass on label definitions and reference-doc generation for them
* Tune labels
* Drop the 'csam' label in favor of using !hide
CSAM is an extremely delicate issue. Moderators must be careful to
remove it from the system without calling direct attention to it.
Using the generic !hide achieves the same effect while obscuring
the reason for the removal. Server and AppView bans will then be
employed (along with any other needed mechanisms) to strip the
content from the network.
There is also a 'false accusation' concern with using the csam
label. Fingerprinting techniques and AI are used to scan for csam.
These are capable of producing false positives which will then be
reverted after human review. The reputational damage caused by a
false positive could be severe, so we want to be careful about any
system which might publicize a false positive.
* Add label definition code generation
* Implement all type signatures for moderation
* Complete a first pass on the labeling sdk
* Add post-moderation test suite
* Add post moderation behavior documentation
* Add self-post behaviors
* Improve post moderation doc
* Tune up the post moderation doc
* Tune up the post moderation doc
* Tune up the post moderation doc
* Simplify the post moderation behavior descriptions
* More behavior description tuneup
* Add profile moderation final behaviors and tests
* Improve generated post-moderation-behaviors doc
* Add profile moderation behaviors doc
* Test muted-by-list specifically
* Fixes to label descriptions
* Fix to muted-by-list behavior
* Dont blur account in moderateProfile() when muting
* Tune label copy
* Apply post embed blurring when account is marked with blurmedia label
* Fix output signature
* Fixes to blocking behavior
* Rename LabelDefinitionPreference to LabelPreference
* Update docs
* Fix test
* Fix: self-harm should blur media only
* Fixes to tests
* Improve label copy
* Remove all labels that do not have a specific policy
Communicating moderation policies with active users is important.
This PR originally included labels which were proposed but did not
yet have policies. While we didn't intend to use them until a
policy was established, I decided it's better to hold off putting
them in the code until we're sure about them. They can be found in
backup files prefixed with "proposed-".
Matthieu Sieben
bnewbold
Paul Frazee
Daniel Holmgren
Dimitris Apostolou
Eric Bailey