* Improve error message when using invalid client_id during code exchange
* Extract SPA example OAuth client in own package
* wip
* remove dependency on get-port
* Properly configure jest to only transpile "get-port" from node_modules
https://jestjs.io/docs/configuration#transformignorepatterns-arraystring
* Use dynamically assigned port number during tests
* use puppeteer to run tests
* remove login input "id" attribute
* code style
* add missing declaration
* tidy
* headless
* remove get-port dependency
* fix tests/proxied/admin.test.ts
* fix tests
* Allow unsecure oauth providers through configuration
* transpile "lande" during ozone tests
* Cache Puppeteer browser binaries
* Use puppeteer cache during all workflow steps
* remove use of set-output
* use get-port in xrpc-server tests
* Renamed to allowHttp
* tidy
* tidy
* Codegen
* Explicitly add Zod (already a peer dep) and validation to api
* Add Nux methods
* Match naming convention
* Remove id, it won't be used
* Add tests
* Use id instead of name, little clearer
* Update API contracts
* Update tests
* Changeset
* Don't mutate
* feat(api): support creation of oauth based AtpAgents
* oauth: misc fixes for confidential clients
* fix(xprc): remove ReadableStream.from polyfill
* OAuth docs tweaks (#2679)
* OAuth: clarification about client_name being shown
* OAuth: re-write handle resolution privacy concern
* avoid relying on ReadableStream.from in xrpc-server tests
* feat(oauth-types): expose "ALLOW_UNSECURE_ORIGINS" constant
* feat(handle-resolver): expose "AtprotoIdentityDidMethods" type
* fix(oauth-client): ensure that the oauth metadata document contains client_id_metadata_document_supported
* fix(oauth-types): prevent unknown query string in loopback client id
* fix(identity-resolver): check that handle is in did doc's "alsoKnownAs"
* feat(oauth-client:oauth-resolver): allow logging in using either the PDS URL or Entryway URL
* fix(oauth-client): return better error in case of invalid "oauth-protected-resource" status code
* refactor(did): group atproto specific checks in own
* feat(api): relax typing of "appLabelers" and "labelers" AtpClient properties
* allow any did as labeller (for tests mainly)
* fix(api): allow to override "atproto-proxy" on a per-request basis
* remove release candidate versions from changelog
* update changeset for api and xrpc packages
* Add missing changeset
* revert RC versions
* Proper wording in OAUTH.md api example
* remove "pre" changeset file
* xrpc: restore original behavior of setHEader and unsetHeader
* docs: add comment for XrpcClient 's constructor arg
* feat(api): expose "schemas" publicly
* feat(api): allow customizing the whatwg fetch function of the AtpAgent
* docs(api): improve migration docs
* docs: change reference to BskyAgent to AtpAgent
* docs: mention the breaking change regarding setSessionPersistHandler
* fix(api): better split AtpClient concerns
* fix(xrpc): remove unused import
* refactor(api): simplify class hierarchu by removeing AtpClient
* fix(api): mock proper method for facets detection
* restore ability to restore session asynchronously
* feat(api): allow instantiating Agent with same argument as super class
* docs(api): properly extend Agent class
* style(xrpc): var name
* docs(api): remove "async" to header getter
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Hailey <me@haileyok.com>
* Sketch proposal for additional muted words attributes
* Rename ttl -> expiresAt
* Feedback
* Codegen
* Refactor muted words methods to integrate new attributes
* Add changeset
* Use datetime format
* Simplify migration
* Fix tests
* Format
* Re-integrate tests
* Let the lock cook
* Fix comments
* Integrate mute words enhancements (#2643)
* Check expiry when comparing mute words
* Check actors when comparing
* Tweak lex, condegen
* Integrate new prop
* Remove fake timers
(cherry picked from commit ad31910560ce938e3ff64944d46355c64635ebf8)
* Update changeset
* Prevent deleting value when updating
* Include missing test
* Add default
* Apply default 'all' value to existing mute words to satisfy Typescript
* Fix types in tests
* Fix types on new tests
* Add bskyAppState preference and api for dismissed nudges
* Add tests
* Wrap preference updates in an lock region to avoid clobbering updates
* Move to queuedNudges and activeProgressGuide
* Fix type error
* Update test
* Add changeset
* chore(deps): update zod
* chore(deps): update pino to match entryway version
* chore(tsconfig): remove truncation of types through noErrorTruncation
* add support for DPoP token type when logging
* fix(bsky): JSON.parse does not return value of type JSON
* fix(pds): add res property to ReqCtx
* fix(pds): properly type getPreferences return value
* chore(tsconfig): disable noFallthroughCasesInSwitch
* refactor(pds): move tracer config in own file
* feat(dev-env): start with "pnpm dev"
* feat(oauth): add oauth provider & client libs
* feat(pds): add oauth provider
* chore: changeset
* feat: various fixes and improvements
* chore(deps): update better-sqlite3 to version 10.0.0 for node 22 compatibility
* chore(deps): drop unused tslib
* fix(did): normalize service IDs before looking for duplicates
* fix(did): avoid minor type casting
* fix(did): improve argument validation
* fix(fetch): explicit use of negation around number comparison
* fix(oauth-provider): improve argument validation
* feat(did): add ATPROTO specific "isAtprotoDidWeb" method
* feat(rollup-plugin-bundle-manifest): add readme
* feat(lint): add eqeqeq rule (only allow == and != with null)
* fix(oauth-client-browser): typo in gitignore
* fix(oauth-provider): properly name error class file
* fix(oauth-provider): remove un-necessary useMemo
* fix(did-resolver): properly build did:web document url
* fix(did-resolver): remove unused types
* fix(fetch): remove unused utils
* fix(pds): remove unused script and dependency
* fix(oauth-provider): simplify isSubPath util
* fix(oauth-provider): add InvalidRedirectUriError static constructor
* fix(jwk): improve JWT validation to provide better error messages and distinguish between signed and unsigned tokens
* fix(pds): use "debug" log level for fetch method
* fix(pds): allow access tokens to contain an unknown "typ" claim (with the exception of "dpop+jwt")
* fix(jwk): remove un-necessary code
* fix(pds): account for whitespace chars when checking JSON
* fix(pds): remove oauth specific config
* fix(pds): run all write queries through transaction or executeWithRetry
fix(pds): remove outdated comments
fix(pds): rename used_refresh_token columns & added primary key
fix(pds): run cleanup task through backgroundQueue
fix(pds): add device.id foreign key to device_account
fix(pds): add comment on cleanup of used_refresh_token
fix(pds): add primary key on device_account
* fix(oauth-provider:time): simplify constantTime util
* fix(pds): rename disableSsrf into disableSsrfProtection
* fix(oauth-client-react-native): remove incomplete package
* refactor(pds): remove status & active from ActorAccount
* fix(pds): invalidate all oauth tokens on takedown
* fix(oauth-provider): enforce token expiry
* fix(pds): properly support deactivated accounts
* perf(pds:db): allow transaction function to be sync
* refactor(psq:account-manager): expose only query builders & data transformations utils from helpers
* fix(oauth-provider): imports from self
* fix(ci): add nested packages to build artifacts
* style(fetch): rename TODO into @TODO
* style(rollup-plugin-bundle-manifest): remove "TODO" from comment
* style(oauth-client): rename TODO into @TODO
* style(oauth-provider): rename TODO into @TODO
* refactor(oauth-client): remove "OAuth" prefix from types
* fix(oauth-client-browser): better type SessionListener
* style(oauth): rename TODO into @TODO
* fix(oauth-provider): enforce provider max session age
* fix(oauth-provider): check authentication parameters against all client metadata
* fix(api): tests
* fix(pds): remove .js from imports for tests
* fix(pds): change account status to match tests
* chore(deps): make all packages depend on the same zod version
* fix(common-web): remove un-necessary binding of Checkable to "zod"
* refactor(jwk): infer jwt schema from refinement definition
* fix(handle-resolver): allow resolution errors to propagate
docs(handle-resolver): better handling of DNS resolution errors
fix(handle-resolver): properly handle DOH responses
* fix(did): service endpoint arrays must contain "one or more" element
* refactor(pipe): simplify implementation
* fix(pds): add missing DB indexes
* feat(oauth): Resolve Authorization Server URI through Protected Resource Metadata
* style:(oauth-client): import order
* docs(oauth-provider:redirect-uri): add reference url
* feat(oauth): implement "OAuth Client ID Metadata Document" from draft-parecki-oauth-client-id-metadata-document-latest internet draft
* feat(oauth-client): backport changes from feat-oauth-client
* docs(simple-store): improve comments
* feat(lexicons): add iterable capabilities
* fix(pds): type error in dev mode
* feat(oauth-provider): improved error reporting
* fix(oauth-types): allow insecure issuer during tests
* fix(xrpc-server): allow upload of empty files
* fix: lint
* feat(fetch): keep request reference in errors
feat(fetch): utilities improvements
* fix(pds): allow more than one session token per user
* feat(ozone): improve env validation error messages
* fix(oauth-client): account for DPoP when checking for invalid_token errors
* fixup! feat(fetch): keep request reference in errors feat(fetch): utilities improvements
* fixup! feat(fetch): keep request reference in errors feat(fetch): utilities improvements
* fix(oauth): various validation fixes
feat(oauth): share client_id validation and parsing utilities between client & provider
* feat(dev-env): fix ozone port number
* fix(fetch-node): prevent fetch against invalid domain names
* fix(oauth-provider): add typings for psl dep
* feat(jwk): make type def compatible with TS 4.x
* fix(oauth): fixed various spec compliance
fix(oauth): return "sub" in refresh token response
fix(oauth): limit token validity for third party clients
fix(oauth): hide client image when not trusted
* fix(oauth): lint
* pds: switch changeset to patch, no breaking changes
* changeset and config for new oauth deps
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* always show your post even if it's labeled
still apply to `contentMedia`
update tests
only apply to posts and `contentView`
add a comment
always show your post even if it's labeled
* Tweak to use a more consistent approach
* Add changeset
---------
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
* Rename bsky_labeler_did
* Use labeldef default setting
* Add definedBy to interpretted label defs
* Improve dev-env mocks for labels
* Remove global label defs for dmca-violation, doxxing, and !no-promote
* Change nudity global label def to default to ignore and no longer be adult-only
* Remove old !no-promote tests
* Add mod authorities header and move bsky labeler into it
* Rename modsPref and addModService/removeModService to labelersPref and add/removeLabeler
* Add defaultSetting and adultOnly to custom label value definitions
* Rename InterprettedLabelValueDefinition to InterpretedLabelValueDefinition
* Update dev-env mock
* Move muted words and hidden posts into moderationPrefs
* Add muted word and hidden post handling to moderatePost
* Add mutewords tests
* Finish muteword tests
* Add mod-authority.test to dev-env
* Rename global label value def gore to graphic-media
* Fix typo
* Stop converting old label values
* Update label target=profile behaviors: dont filter content on hide, dont blur display names, but do show the info cards
* Update label target=account behaviors: dont blur media of content
* Add muteword moderation behaviors
* Fix mock label defs
* Implement quote-post moderation handling
* Add adult content test
* Handle sync legacy labels (#2291)
* Handle sync legacy labels
* Remap values on read
* Filter out double-written legacy label values
* Better naming, fix types
* Fix test
* Update moderation docs in sdk
* Update to new atproto-accept-labelers header behavior
* Add getLabelDefinitions() helper method
* Add proxy header support to agent
* Update mock moderation
* lint
* Implement moderation for userlists and feedgens
* Add another test label
* fix labeler in dev-env agents
* fix label hydration test
* fix lint error
* fix agent test
* fix takedown labels test
---------
Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: dholms <dtholmgren@gmail.com>
* adding some basic views
* feed hydration, add blocks to post hydration
* pass over notification hydration
* tidy
* merge
* implement getProfile
* hydrate post aggregation and viewer state
* fix
* fix codegen
* get some tests passing!
* add takedowns & some like bugfixing
* all profile tests passing!
* likes test
* follow endpoints using data plane
* reorg follow block rules
* reposts
* post views!
* implement getList w/ dataplane caveat
* adjust dataplane getListMembers to return listitem uris
* implement getListMutes and -Blocks w/ dataplane
* suggestions
* timeline
* misc view fixes
* view fixes for mutes, self-mute/block
* author feed
* feed gen routes
* tidy
* misc block/mute fixes
* list feed & actor likes
* implement getLists, fix some empty cursors
* implement getMutes, empty profile description fix
* implement getBlocks, block application fix
* implement getSuggestedFollowsByActor, needs some fixes
* feed generation
* search routes
* threads
* tidy
* fix some snaps
* fix getSuggestedFollowsByActor
* implement listNotifications
* implement getUnreadCount w/ dataplane
* implement notifications.updateSeen w/ dataplane
* 3rd party blocking tests
* blocked profile viewer
* add search mocks
* refactor getFeed
* createPipeline -> createPipelineNew
* basic replygating functionality on dataplane w/o filtering violating replies
* hack threadgates into dataplane, apply gates
* deterministic thread orders in dataplane
* misc cleanup around dataplane
* upgrade typescript to v5.3
* update typescript linter deps
* sync bsky proto, codegen
* update dataplane, sync with bsky proto updates
* remove indexer, ingester, daemon, moderation services from appview
* convert more bsky internals to dataplane, remove custom feedgens, implement mute/unmuting in mock dataplane
* remove bsky services. db and indexing logic into mock dataplane.
* remove tests not needed by appview v2, misc reorg
* add basic in-mem repo subscription to dataplane mock
* fix dev-env, bsky tests, bsky build
* cull bsky service entrypoint
* add bsky service readme
* build
* tidy
* tidy, fix pds proxy tests
* fix
* fix bsky entrypoint deps
* support http2 grpc client
* build
* fix dataplane bad tls config/default
* support multiple dataplane urls, retry when unavailable
* build
* tidy/fix
* move dataplane mock tests into their own dir
* cover label hydration through timeline test
* bring back labels in appview tests
* remove unused db primary/replica/coordinator from bsky dataplane
* bsky proto add cids to contracts, buf codegen
* sync-up bsky data-plane w/ codegen updates
* start using dataplane interaction endpoints
* add file
* avoid overfetching from dataplane, plumb feed items and cids
* pass refs through for post viewer state
* Lexicons: Add labeler prefs, labeler declaration, and get labeler routes
* Add labelerViewBasic and update embed views
* Fix typo
* switch list feeds to use feed item in dataplane
* handle not found err on get-thread dataplane call
* support use of search service rather than dataplane methods
* mark some appview v2 todos
* tidy
* still use dataplane on search endpoints when search service is not configured
* fix pds test
* Switch to labelerViewDetailed
* Move label and report values to refs
* Add getActorLabelers
* lint
* fix up bsky tests & snaps
* small diff to open pr
* rm new line
* codegen schemas
* tidy migrations
* table + indexing
* protos
* rename lexicons
* views, hydration + rename lexicons
* rest of routes
* data plane routes
* parse labelers from req
* fix appview-v2 docker build
* Support label issuer tied to appview v2 (#2033)
support label issuer tied to appview
* hydrate context
* tidy header logic
* integrating into more routes
* more routes
* wrap up rest
* add mock labeler
* rework labelerlexicons
* tidy lexs
* codegen new lexicons
* integrate lexicon rework
* add proxy logic
* forward labeler headers through pds
* tweak label header parsing
* remove did from scheams
* update indexing for lexs
* tests for mod service views
* label hydration test
* Add 'associated' to profileViewDetailed
* Rename labelers to mods in preferences
* Change uri to did in mod preferences
* couple more
* syntax tweaks
* integrate updated lexicons
* update view snap
* handle mod service embeds
* tidy
* fix mock
* lint
* base default labels of config var
* fix label hydration
* Appview v2: handle empty cursor on list notifications (#2017)
handle empty cursor on appview listnotifs
* Update appview v2 to use author feed enum (#2047)
* update bsky protos with author feed enum, misc feed item changes
* support new author feed enums in dataplane
* fix build
* Appview v2: utilize sorted-at field in bsky protos (#2050)
utilize new sorted-at field in bsky protos
* remove all dataplane usage of GetLikeCounts, switch to GetInteractionCounts
* Appview v2, sync w/ changes to protos (#2071)
* sync bsky protos
* sync-up bsky implementation w/ proto changes
* Appview v2 initial implementation for getPopularFeedGenerators (#2072)
add an initial implementation for getPopularFeedGenerators on appview v2
* merge
* fixes
* fix feed tests
* fix bsync mock
* format
* remove unused config
* fix lockfile
* another lockfile fix
* fix duplicate type
* fix dupplicate test
* Appview v2 handling clearly bad cursors (#2092)
* make mock dataplane cursors different from v1 cursors
* fail open on clearly bad appview cursors
* fix pds appview proxy snaps
* Appview v2 no notifs seen behavior (#2096)
* alter behavior for presenting notifications w/ no last-seen time
* fix pds proxy tests
* Appview v2 dataplane retries based on client host (#2098)
choose dataplane client for retries based on host when possible/relevant
* don't apply negated labels
* display suspensions on actor profile in appview v2
* Appview v2 use dataplane for identity lookups (#2095)
* update bsky proto w/ identity methods
* setup identity endpoints on mock dataplane
* move from idresolver to dataplane for identity lookups on appview
* tidy
* Appview v2: apply safe takedown refs to records, actors (#2107)
apply safe takedown refs to records, actors
* Fix timing on appview v2 repo rev header (#2113)
fix timing on appview repo rev
* fix post thread responses
* Appview v2 don't apply 3p self blocks (#2112)
do not apply 3p self-blocks
* Appview v2 search for feed generators (#2118)
* add protos for feedgen search
* support feed search on getPopularFeedGenerators
* Appview v2 config tidy (#2117)
* remove mod and triage roles from appview
* rename cdn and search config
* remove custom feed harness from appview v2
* Appview v2: don't apply missing modlists (#2122)
* dont apply missing mod lists
* update mock dataplane
* Update packages/bsky/src/hydration/hydrator.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* refactor & document a bit better
* fix up other routes
---------
Co-authored-by: devin ivy <devinivy@gmail.com>
* Appview v2 enforce post thread root boundary (#2120)
* enforce post thread root boundary
* test thread root boundary
* Appview v2 fix admin environment variable (#2137)
fix admin env in appview v2
* Remove re-pagination from getSuggestions (#2145)
* remove re-pagination from getSuggestions
* fix test
* Adjust wording for account suspension (#2153)
adjust wording for account suspension
* Appview v2: fix not-found and blocked uris in threads (#2201)
* fix uris of not-found and blocked posts in threads
* update snaps
* ✨ Show author feed of takendown author to admins only (#2197)
* fold in cid, auth, tracing, node version changes
* remove dead config from bsky service entrypoint
* build
* remove ozone test codepaths for appview v2
* tidy, docs fix
* fix test
* add additional user counts
* add associated data to profiles
* update snaps
* update to is_mod_service
* format
* tidy
* 3p labeler sdk updates (#2176)
* Update sdk to support 3p labeler preferences
* Stick with intolerance instead of hate for the label group id
* wip expand labels and label groups
* Output moderationOpts (computed) and modsPref (unaltered)
* Add tests for enabling/disabling mod services
* Add atproto-labelers header config
* Expand labels and label groups in definitions
* Fix tests
* Tweaks to labels
* Remove label descriptions and improve output types on labels and label groups
* Add mocker to exported API
* Improve types of label and label group definitions
* Rework moderation prefs to continue using global labelgroup settings and only disable label groups per moderator
* Simplify encoding of the label preferences in definitions
* Add target constraints to labels
* Refactor the moderation sdk to derive more behaviors from the definition files
* Small cleanup
* Add hiding tool to modsdk
* Track filter causes
* Make mute state an alert
* Fix: dont blur profileview for blocks
* Prioritize causes by severity
* Add moderateNotification() and drop quote post moderation code
* Add mocker functions for notifications
* Improve mock data
* Lexicon: Add custom label definitions and remove modservice descriptions
* Lexicon: Update moderation prefs
* SDK updates: remove label groups, reduce builtin labels, update mod-preference apis
* Lexicon: Update global labels with new reduced set
* Lexicon: Remove moderation.getService and add detailed option to getServices
* Lexicons: add severity=none to custom label value defs
* Implement custom label-value definition tooling
* All custom labels are no-self
* Backend impl for labeler lexicon updates (#2258)
* codegen
* clean up impl
* fix up tests
* Lexicon: modservice -> labeler
* Remove x- prefix behavior; add label value syntax rules; add custom label precedence rules
* Lexicon: Remove the ability to choose a defaultSetting from custom labels
* Rework test suites
* Give behaviors to all labels regardless of target
* sync up backend with lex changes
* fix labelers in dev-env agent
* lint protos
* update protos & views
* small bugfix & update tests
* tweak protos
* fix build issue from merge
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
* Allow tags to lead with and contain only numbers
* Break tags on other whitespace characters
* Export regexes from rich text detection
* Add test
* Add test
* Disallow number-only tags
* Avoid combining enclosing screen chars
* Allow full-width number sign
* Clarify tests
* Fix punctuation edge case
* Reorder
* Simplify, add another test
* Another test, comment
* Don't remove hash from muted words
* Split out crud actions, only sanitize on inserts
* Add changeset
* Handle hash emoji in mute words
* Add sanitization for invalid chars
* Remove console
* Add util
* Clean up changesets
* Format
* Wow forgot to commit change
* Do not parse hashtag emoji as tag (#2242)
* fix: prevent hashtag emoji from being parsed as tag
* chore: fmt
* fix: properly calculate length of tag
* Add a couple tests
---------
Co-authored-by: Mary <148872143+mary-ext@users.noreply.github.com>
* draft of account migration lexicons
* format
* clean up schemas
* codegen
* second pass on schemas
* small fix
* move around checkImportStatus
* re-codegen
* getServiceAuth
* getServiceAuth impl
* importRepo impl
* handle uploadBlob for import
* allow bringing your own did on createAccount
* working on test flow
* fleshing out flow
* fix up sendPlcOP
* small fixes
* activate/deactivate account
* full flow working!
* schema tweaks
* format
* update schemas
* moar codegen
* match impl to new schemas
* email flow for signed plc operation
* add email flow for plc operations
* impl plc op request email
* proxy to entryway
* tidy activate account
* integrating account deactivated state
* fix up tests
* friendly parse on optional did auth
* admin activate/deactivate routes
* proxy relevant requests to entryway
* remove admin activation routes
* do not proxy acitvaition to entryway
* cfg for disallowing imports
* buff up test
* refactor listMissingBlobs a bit
* add validDid & activated to accoutn status
* emit event on account activation
* test creating a post after migrating
* account deactivation tests
* test name
* tests on plc operations
* fix recommended did creds
* codegen
* turn off accepting imports on createAccount
* undo prev change
* increment version
* build branch
* pr feedback
* handle errs in p-queue
* handle blob upload outside of txn
* Clean old temp account migration lexicons (#2187)
* clean old temp lexicons
* rm old test
* fix agent session test
* fix bsky test
* dont build branch
* Better handling of resumeSession errors
* Update test
* Format
* Pare back to only necessary
* Update handling for 500s
* Should really be update
* Update logic from feedback
* Update tests
* Feedback
* Revert debug change
* Changeset
* Bump minor
* add optional did doc output to getSession lexicon
* add did doc output to getSession on pds, update client to use it
* api test fixes
* api changeset
* tidy
* lexicon for did doc w/ auth credentials
* include did doc w/ session when configured. configure on dev-env.
* Add dynamic PDS URL adoption to the client
* remove usage of did doc field from getsession in client
* dry-up did doc type and validation
* remove explicit dep on zod by identity package
* move more did doc parsing to common-web
* go back to strings
* rollback breaking changes to identity package
* add changeset
---------
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
Co-authored-by: dholms <dtholmgren@gmail.com>
* make lint
* Update packages/api/tests/rich-text-detection.test.ts
Co-authored-by: Eric Bailey <git@esb.lol>
---------
Co-authored-by: Eric Bailey <git@esb.lol>
* rm tables
* rm event-stream & proxied
* Remove appview services, move label service to pds
* only proxy appview stuff
* delete more tables
* Start removing message dispatched from pds
* more syncing-up removal of message dispatcher in pds
* merged
* remove feedgens from pds, remove getPopular
* remove unused image helper from pds
* fixing compiler errors
* clean up sharp
* rm label service
* first pass on cleaning up tests
* fix up a bunch of tests
* moderation view tests
* last admin tests
* got a lil overzealous in deletes
* clean up unused cfg
* clean up label table
* simplify admin repo search query/logic
* tidy pds entrypoint
* in-progress pds config changes
* cfg fiddling
* finish cleaning up cfg/ctx
* comments
* building
* pds prefix on env
* test env
* collapse pds migrations down into a single migration
* fix up dev-env
* tidy
* cleanup
* fix pds admin tests
* fix handle test
* fix pds proxy tests
* fix subscribe repos test
* fix sqlite config in pds tests
* add sqlite clause in sequencer-leader
* fix actor search w/ sqlite on pds
* fixes
* fix dev env build
* update pds service entrypoint
* simple env example
* make takedown ids opaque identifiers in the pds
* use pds routes for api tests
* update pds dockerfile with volume and correct port env var
* add a couple env vars to example
* add comments to env example
* @atproto/pds 0.2.0-beta.0
* @atproto/aws 0.0.1-beta.0
* appview did
* @atproto/aws 0.0.1
* enable logs by default
* update env example
* bugfixing sandbox issues
* consistency in pds env var name for appview url
* log on pds start and stop, configure version at runtime
* @atproto/pds 0.2.0-beta.1
* fix semver matching for pds beta version
* v0.2.0-beta.2
* default invites to being not required
* fix flaky test
* limit db connections in tests
* publish 0.2.0-beta.d3
* fix invite required parsing
* @atproto/pds 0.2.0-beta.5
* Proxy getPopularFeedGenerators on simplified pds (#1222)
proxy getPopularFeedGenerators on pds
Co-authored-by: dholms <dtholmgren@gmail.com>
* tidy migrations
* fix service entry
* bump version
* change auth order
* bump version
* bump version
* add upgradeRepoVersion & fallback url for cdn
* bump version
* merging
* merge pds
* building dev-env
* merging tests
* merge service entry
* test fixing
* tidy
* fix admin search
* tidy
* tidy
* add snap for getListFeed
* add backup nameserver cfg
* tidy + pr feedback
* tidy
* tidy env
* bit more
* re-add dotenv to root package.json
* fix dep
* build branch
* fix tests
* Refactor tests to make better use of dev-env (#1690)
* refactor pds tests to use dev env
* refactor bsky tests
* fix pds test
* tidy bsky tests
* build pds correctly
* fix entry point
* default logging to false (for now)
* format service entry
* Switch takedown ids back to ints on pds distribution (#1694)
* switch takedown ids back to ints, consistent with live pds
* tidy/fix migration
* update migration for sqlite
* export moderation action reversal
* takedown tests
* dont build branch
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* lexicons
* codegen
* email templates
* request routes
* impl
* migration
* tidy
* tests
* tidy & bugfixes
* format
* fix api test
* fix auth test
* codegen
* add unique constraint
* Add email confirmed to AtpSessionData
* interop test files (#1529)
* initial interop-test-files
* crypto: switch signature-fixtures.json to a symlink
* syntax: test against interop files
* prettier
* Update interop-test-files/README.md
Co-authored-by: Eric Bailey <git@esb.lol>
* disable prettier on test vectors
---------
Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: dholms <dtholmgren@gmail.com>
* add getSuggestedFollowsByActor (#1553)
* add getSuggestedFollowsByActor lex
* remove pagination
* codegen
* add pds route
* add app view route
* first pass at likes-based suggested actors, plus tests
* format
* backfill with suggested_follow table
* combine actors queries
* fall back to popular follows, handle backfill differently
* revert seed change, update test
* lower likes threshold
* cleanup
* remove todo
* format
* optimize queries
* cover mute lists
* clean up into pipeline steps
* add changeset
* List feeds (#1557)
* lexicons for block lists
* reorg blockset functionality into graph service, impl block/mute filtering
* apply filterBlocksAndMutes() throughout appview except feeds
* update local feeds to pass through cleanFeedSkeleton(), offload block/mute application
* impl for grabbing block/mute details by did pair
* refactor getActorInfos away, use actor service
* experiment with moving getFeedGenerators over to a pipeline
* move getPostThread over to a pipeline
* move feeds over to pipelines
* move suggestions and likes over to pipelines
* move reposted-by, follows, followers over to pipelines, tidy author feed and post thread
* remove old block/mute checks
* unify post presentation logic
* move profiles endpoints over to pipelines
* tidy
* tidy
* misc fixes
* unify some profile hydration/presentation in appview
* profile detail, split hydration and presentation, misc fixes
* unify feed hydration w/ profile hydration
* unify hydration step for embeds, tidy application of labels
* setup indexing of list-blocks in bsky appview
* apply list-blocks, impl getListBlocks, tidy getList, tests
* tidy
* update pds proxy snaps
* update pds proxy snaps
* fix snap
* make algos return feed items, save work in getFeed
* misc changes, tidy
* tidy
* fix aturi import
* lex
* list purpose
* lex gen
* add route
* add proxy route
* seed client helpers
* tests
* mutes and blocks
* proxy test
* snapshot
* hoist actors out of composeThread()
* tidy
* tidy
* run ci on all prs
* format
* format
* fix snap name
* fix snapsh
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* Improve xrpc server error handling (#1597)
improve xrpc server error handling
* Remove appview proxy runtime flags (#1590)
* remove appview proxy runtime flags
* clean up proxy tests
* getPopular hotfix (#1599)
dont pass all params
* Interaction Gating (#1561)
* lexicons for block lists
* reorg blockset functionality into graph service, impl block/mute filtering
* apply filterBlocksAndMutes() throughout appview except feeds
* update local feeds to pass through cleanFeedSkeleton(), offload block/mute application
* impl for grabbing block/mute details by did pair
* refactor getActorInfos away, use actor service
* experiment with moving getFeedGenerators over to a pipeline
* move getPostThread over to a pipeline
* move feeds over to pipelines
* move suggestions and likes over to pipelines
* move reposted-by, follows, followers over to pipelines, tidy author feed and post thread
* remove old block/mute checks
* unify post presentation logic
* move profiles endpoints over to pipelines
* tidy
* tidy
* misc fixes
* unify some profile hydration/presentation in appview
* profile detail, split hydration and presentation, misc fixes
* unify feed hydration w/ profile hydration
* unify hydration step for embeds, tidy application of labels
* setup indexing of list-blocks in bsky appview
* apply list-blocks, impl getListBlocks, tidy getList, tests
* tidy
* update pds proxy snaps
* update pds proxy snaps
* fix snap
* make algos return feed items, save work in getFeed
* misc changes, tidy
* tidy
* fix aturi import
* initial lexicons for interaction-gating
* add interactions view to post views
* codegen
* model bad reply/interaction check state on posts
* initial impl for checking bad reply or interaction on write
* omit invalid interactions from post thread
* support not-found list in interaction view
* hydrate can-reply state on threads
* present interaction views on posts
* misc fixes, update snaps
* tidy/reorg
* tidy
* split interaction gating into separate record in lexicon
* switch interaction-gating impl to use separate record type
* allow checking reply gate w/ root post deletion
* fix
* initial gating tests
* tighten gated reply views, tests
* reply-gating list rule tests
* allow custom post rkeys within window
* hoist actors out of composeThread()
* tidy
* update thread gate lexicons, codegen
* lex fix
* rename gate to threadgate in bsky, update views
* lex fix
* improve terminology around reply validation
* fix down migration
* remove thread gates on actor unindexing
* add back .prettierignore
* tidy
* run ci on all prs
* syntax
* run ci on all prs
* format
* fix snap
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* order by `like.indexedAt` in app view (#1592)
* order by like.indexedAt
* use keyset for ordering
* simplify
* ok ok ok I get it now
* Update packages/bsky/src/api/app/bsky/feed/getActorLikes.ts
Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>
---------
Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>
* Remove default value for post table invalid attrs (#1601)
remove default value for post table attrs
* Version packages (#1602)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* update Bluesky PBLLC to PBC (Public Benefit Corporation) (#1600)
* Temporarily disable filtering `invalidReplyRoot`s (#1609)
temporarily disable invalidReplyRoot check
* fix syntax docs (#1611)
* Version packages (#1612)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Allow bypass on ratelimit ip (#1613)
allow bypass on ratelimit ip
* Write rate limits (#1578)
* get rate limit ip correctly
* add write rate-limits
* Tweak createSession rate limit key (#1614)
tweak create session rl key
* Filter preferences for app passwords (#1626)
filter preferences for app passwords
* Tweak rate limit setup for multi rate limit routes (#1627)
tweak rate limit setup for multi rate limit routes
* Remove zod from xrpc-server error handling (#1631)
remove zod from xrpc-server error handling check
* Enforce properties field on lexicon object schemas (#1628)
* add empty properites to thread gate schema fragments
* tweak lexicon type
* Add feed-vew and thread-view preferences (#1638)
* Add feed and thread preference lexicons
* Add feed-view and thread-view preference APIs
* Add changeset for new preferences (#1639)
Add changeset
* Version packages (#1640)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Disable getAccountInviteCodes for app passwords (#1642)
disable getAccountInviteCodes for app passwords
* remove cruft packages (uri, nsid, identifier) (#1606)
* remove @atproto/nsid (previously moved to syntax)
* remove @atproto/uri (previously moved to syntax)
* remove @atproto/identifier (previously moved to syntax)
* bump lockfile to remove old packages
---------
Co-authored-by: Eric Bailey <git@esb.lol>
* api: update login/resumeSession examples in README (#1634)
* api: update login/resumeSession examples in README
* Update packages/api/README.md
Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>
---------
Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>
* small syntax lints (#1646)
* lint: remove unused imports and variables
* lint: prefix unused args with '_'
* eslint: skip no-explicit-any; ignore unused _var (prefix)
* eslint: explicitly mark ignores for tricky cases
* indicate that getPopular is deprecated (#1647)
* indicate that getPopular is deprecated
* codegen for deprecating getPopular
* tidy up package.json and READMEs (#1649)
* identity: README example and tidy
* tidy up package metadata (package.json files)
* updated README headers/stubs for several packages
* crypto: longer README, with usage
* syntax: tweak README
* Apply suggestions from code review
Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: devin ivy <devinivy@gmail.com>
---------
Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: devin ivy <devinivy@gmail.com>
* Improve the types of the thread and feed preferences APIs (#1653)
* Improve the types of the thread and feed preferences APIs
* Remove unused import
* Add changeset
* Version packages (#1654)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Disable pds appview routes (#1644)
* wip
* remove all canProxyReadc
* finish cleanup
* clean up tests
* fix up tests
* fix api tests
* fix build
* fix compression test
* update image tests
* fix dev envs
* build branch
* fix service file
* re-enable getPopular
* format
* rm unused sharp code
* dont build branch
* auto-moderator tweaks: pass along record URI, create report for takedown action (#1643)
* auto-moderator: include record URI in abyss requests
* auto-moderator: log attempt at hard takedown; create report as well
The motivation is to flag the event to mod team, and to make it easier
to confirm that takedown took place.
* auto-mod: typo fix
* auto-mod: bugfixes
* bsky: always create auto-mod report locally, not pushAgent (if possible)
* bsky: fix auto-mod build
* bsky: URL-encode scanBlob call
* Clear follow viewer state when blocking (#1659)
* clear follow viewer state when blocking
* tidy
* add `tags` to posts (#1637)
* add tags to post lex
* kiss
* add richtext facet and validation attrs
* add tag validation attrs to post
* codegen
* add maxLength for tags, add description
* validate post tags on write
* add test
* handle tags in indexer
* add tags to postView, codegen
* return tags on post thread view
* format
* revert formatting change to docs
* use establish validation pattern
* add changeset
(cherry picked from commit fcb6fe7c26144662f791c7900afcd84c7bf1be6b)
* remove tags from postView, codegen
* remove tags from thread view
* revert unused changes
* Version packages (#1664)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* merge
* Reverse order of blocks from sync.getRepo (#1665)
* reverse order of blocks from sync.getRepo
* write to car while fetching next page
* Add hashtag detection to richtext (#1651)
* add tag detection to richtext
* fix duplicate tag index error
* add utils
* fix leading space index failures, test for them
* add changeset
* Version packages (#1669)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* proposed new search lexicons (#1594)
* proposed new search lexicons
* lexicons: lint
* lexicons: fix actors typo
* lexicons: camelCase bites again, ssssss
* lexicons: add 'q' and mark 'term' as deprecated for search endpoints
* codegen for search lexicon updates
* bsky: prefer 'q' over 'term' in existing search endpoints
* search: bugfix
* lexicons: make unspecced search endpoints return skeleton obj
* re-codegen for search skeleton obj
* Disable pds appview indexing (#1645)
* rm indexing service
* remove message queue & refactor background queue
* wip
* remove all canProxyReadc
* finish cleanup
* clean up tests
* fix up tests
* fix api tests
* fix build
* fix compression test
* update image tests
* fix dev envs
* build branch
* wip - removing labeler
* fix service file
* remove kysely tables
* re-enable getPopular
* format
* cleaning up tests
* rm unused sharp code
* rm pds build
* clean up tests
* fix build
* fix build
* migration
* tidy
* build branch
* tidy
* build branch
* small tidy
* dont build
* Refactor PDS appview routes (#1673)
move routes around
* Strip leading `#` from from detected tag facets (#1674)
ensure # is removed from facets
* Version packages (#1675)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Proxy search queries (#1676)
* proxy search
* tweak profile resp
* fix admin.searchRepos
* add mock mailer
* Fix to daniel's MOCKERY of a mock mailer
* Don't allow non-verified email updates until app feature is out (#1682)
stricter updating email until app feature is out
* changesets
---------
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Fix to handle duplicate preference key entries
* Add personal details preference API to sdk
* Add Array.prototype.findLast() type declaration
* Move interface declaration to ensure it's included in other package builds
* Add self-label schemas
* switch around array and union for self-labels
* apply self-labels to post and profile views
* test for self labels in pds
* apply self-labels to post and profile views in bsky appview
* test for self labels in bsky appview
* update pds proxied test snapshots
* Add support for self-labels to the mdoeration sdk
* Disable unknown-labeler filtering until 3P support is added
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* First pass on label definitions and reference-doc generation for them
* Tune labels
* Drop the 'csam' label in favor of using !hide
CSAM is an extremely delicate issue. Moderators must be careful to
remove it from the system without calling direct attention to it.
Using the generic !hide achieves the same effect while obscuring
the reason for the removal. Server and AppView bans will then be
employed (along with any other needed mechanisms) to strip the
content from the network.
There is also a 'false accusation' concern with using the csam
label. Fingerprinting techniques and AI are used to scan for csam.
These are capable of producing false positives which will then be
reverted after human review. The reputational damage caused by a
false positive could be severe, so we want to be careful about any
system which might publicize a false positive.
* Add label definition code generation
* Implement all type signatures for moderation
* Complete a first pass on the labeling sdk
* Add post-moderation test suite
* Add post moderation behavior documentation
* Add self-post behaviors
* Improve post moderation doc
* Tune up the post moderation doc
* Tune up the post moderation doc
* Tune up the post moderation doc
* Simplify the post moderation behavior descriptions
* More behavior description tuneup
* Add profile moderation final behaviors and tests
* Improve generated post-moderation-behaviors doc
* Add profile moderation behaviors doc
* Test muted-by-list specifically
* Fixes to label descriptions
* Fix to muted-by-list behavior
* Dont blur account in moderateProfile() when muting
* Tune label copy
* Apply post embed blurring when account is marked with blurmedia label
* Fix output signature
* Fixes to blocking behavior
* Rename LabelDefinitionPreference to LabelPreference
* Update docs
* Fix test
* Fix: self-harm should blur media only
* Fixes to tests
* Improve label copy
* Remove all labels that do not have a specific policy
Communicating moderation policies with active users is important.
This PR originally included labels which were proposed but did not
yet have policies. While we didn't intend to use them until a
policy was established, I decided it's better to hold off putting
them in the code until we're sure about them. They can be found in
backup files prefixed with "proposed-".
