* feat(api): support creation of oauth based AtpAgents
* oauth: misc fixes for confidential clients
* fix(xprc): remove ReadableStream.from polyfill
* OAuth docs tweaks (#2679)
* OAuth: clarification about client_name being shown
* OAuth: re-write handle resolution privacy concern
* avoid relying on ReadableStream.from in xrpc-server tests
* feat(oauth-types): expose "ALLOW_UNSECURE_ORIGINS" constant
* feat(handle-resolver): expose "AtprotoIdentityDidMethods" type
* fix(oauth-client): ensure that the oauth metadata document contains client_id_metadata_document_supported
* fix(oauth-types): prevent unknown query string in loopback client id
* fix(identity-resolver): check that handle is in did doc's "alsoKnownAs"
* feat(oauth-client:oauth-resolver): allow logging in using either the PDS URL or Entryway URL
* fix(oauth-client): return better error in case of invalid "oauth-protected-resource" status code
* refactor(did): group atproto specific checks in own
* feat(api): relax typing of "appLabelers" and "labelers" AtpClient properties
* allow any did as labeller (for tests mainly)
* fix(api): allow to override "atproto-proxy" on a per-request basis
* remove release candidate versions from changelog
* update changeset for api and xrpc packages
* Add missing changeset
* revert RC versions
* Proper wording in OAUTH.md api example
* remove "pre" changeset file
* xrpc: restore original behavior of setHEader and unsetHeader
* docs: add comment for XrpcClient 's constructor arg
* feat(api): expose "schemas" publicly
* feat(api): allow customizing the whatwg fetch function of the AtpAgent
* docs(api): improve migration docs
* docs: change reference to BskyAgent to AtpAgent
* docs: mention the breaking change regarding setSessionPersistHandler
* fix(api): better split AtpClient concerns
* fix(xrpc): remove unused import
* refactor(api): simplify class hierarchu by removeing AtpClient
* fix(api): mock proper method for facets detection
* restore ability to restore session asynchronously
* feat(api): allow instantiating Agent with same argument as super class
* docs(api): properly extend Agent class
* style(xrpc): var name
* docs(api): remove "async" to header getter
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Hailey <me@haileyok.com>
* cleanup repeat process all
* wip
* skip actor search test
* skip actor search test
* tweak processAll
* decrease wait to 1 sec
* repo_blob -> record_blob
* simplify backlink linkTo
* return repo_root to one row
* sequence before updating repo_root
* invite code forUser -> forAccount
* ipld_block -> repo_block
* use lru-cache fetchMethod
* move did_cache to own db
* better error handling on did cache
* drop did_handle
* fix sequencer wait time
* debug
* debug
* more debug
* check something
* fix bday paradox
* fix bday paradox
* tidy up pds service auth
* rm skipped test
* retry http
* tidy
* improve fanout error handling
* fix test
* return signing key in did-web
* more tests
* tidy serivce auth checks
* user_account -> account
* remove inviteNote
* keypair per repo
* use an lru cache for keypairs as well
* clean up repo
* wip
* wrap up accoutn manager
* tidy
* tidy
* fix tests
* fix disabled codes
* fix appview tests
* add note
* set pragmas
* tidy account manager getDb
* rename pref transactor
* user pref -> account pref
* handle blob imports
* tidy imports
* add reserveSigningKey
* wip transferAccount
* clean up transferAccount
* tests
* tidy
* tidy
* configure entryway url on pds
* handle entryway in pds admin endpoints
* make importRepo temp
* fix imports
* make email optional on pds when using entryway
* handle diffs
* handle pds entryway usage for server, identity, admin endpoints
* pds support for credentials from entryway
* setup pds tests w/ entryway service
* tidy
* tidy
* update entryway version
* wip
* test handle updates w/ entryway
* split account table into two
* tidy
* tweak scripts
* tidy tests
* tidy
* better config for actorstore & dbs
* clean up cfg more
* reorg actorstore fs layout
* handle erros on actor db create
* pr tidy & fix accoutn deletion test
* pr feedback
* fix bad merge
* unskip test
* fix subscribe repos tests
* tidy repo root tables
* tidy
* fix tests
* tidy delete tokens
* tidy account getters
* tidy
* bulk deletesg
* increase chunk size
* handle racing refreshes
* wip
* fix auth test
* invert import flow
* clean up actor store on create account failure
* tweak sequencer
* prevent invite code races on createAccount
* rm note
* add back in race protection on getAccountInviteCodes
* start feature branch
* deleted app migration table
* patch up new auth test
* rm note
* g
* create accoutn delegated from entryway
* tidy
* fix test
* change plcOp type to unknown
* small fixes
* sync up w entryway branch
* Use proper error when authed account is not found (#1799)
provide proper error when account not found in access-takedown check
* build branch
* build on ghcr
* tweak service file
* tweak service file
* change where we save reserved keys
* no tmp dir in blobstore either
* fix blobstore temp location again
* handle repeat record_blobs
* create account before submitting plc op & undo if fail
* small tweak
* limit the number of local records
* push out empty commit on transfer
* fix issue with record_blob
* add push blob endpoint
* Set and validate token audiences on pds v2 (#1793)
set and validate token audience on pds v2
* merge
* include entryway did on tests
* build branch
* fix cache issue
* xrpc server blob limit
* put correct bytes
* add auth to routes
* handle quarantining/unquarantining a blob that does not exist
* tidy
* fix transfer tests
* fix email request routes for entryway
* PDS v2 entryway account deletion (#1819)
* add admin lexicon for account deletion
* implement admin account deletion endpoint
* fix entryway proxying on account email checks
* proxy to entryway for acct deletion
* read-after-write sanity check
* tweak
* wip
* finish refactor
* fix test schema
* application retry logic for busy
* pr feedback
* rm lru-cache
* fix test pg schema
* fix transfer test
* Sqlite instrumentation for pds v2 (#1838)
* sqlite instrumentation
* build
* remove build
* dont reimport blobs
* send ticks during import
* close on error
* catch handle validation error
* add log
* fix test
* return emailConfirmedAt on getAccountInfo
* Upgrade sharp on pds v2 (#1863)
upgrade sharp to 0.32.6
* read all bytes before parsing car
* Async car reader (#1867)
* asynchronously read in car
* dont buffer car
* tweak
* Gracefully handle indexing of invalid records (#1853)
* gracefully handle indexing of invalid records
* fix repo tests
* Fix role auth for access-or-role verifier, getBlob check on actor takedowns (#1869)
fix role auth for access-or-role verifier, fix getBlob actor takedown check
* better cleanup of actor-stores
* add ability to not ensure leaves
* tidy
* allow did:web transfer
* Migration utility for actor-store (#1873)
beginnings of helper for migrating all actors
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* base case for findBlobRefs
* App-level retries for sqlite on pds (#1871)
* revamp retry helper to be more flexible re: backoff strategies
* sqlite timeout helper
* ensure sqlite wal on db creation/migration rather than every open
* layer retries for sqlite on writes outside transactions on pds
* tidy
* fix up lockfile
* tidy
* fix lex codegen
* fix timing bug in threadgate test
* No-op update handling (#1916)
do no produce commits on no-op updates
* Retry on all SQLITE_BUSY error codes (#1917)
retry on all sqlite_busy error codes
* Pds v2 ensure sqlite ready (#1918)
ensure sqlite is ready before making queries
* try something
* tidy
* dont build branch
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* rm tables
* rm event-stream & proxied
* Remove appview services, move label service to pds
* only proxy appview stuff
* delete more tables
* Start removing message dispatched from pds
* more syncing-up removal of message dispatcher in pds
* merged
* remove feedgens from pds, remove getPopular
* remove unused image helper from pds
* fixing compiler errors
* clean up sharp
* rm label service
* first pass on cleaning up tests
* fix up a bunch of tests
* moderation view tests
* last admin tests
* got a lil overzealous in deletes
* clean up unused cfg
* clean up label table
* simplify admin repo search query/logic
* tidy pds entrypoint
* in-progress pds config changes
* cfg fiddling
* finish cleaning up cfg/ctx
* comments
* building
* pds prefix on env
* test env
* collapse pds migrations down into a single migration
* fix up dev-env
* tidy
* cleanup
* fix pds admin tests
* fix handle test
* fix pds proxy tests
* fix subscribe repos test
* fix sqlite config in pds tests
* add sqlite clause in sequencer-leader
* fix actor search w/ sqlite on pds
* fixes
* fix dev env build
* update pds service entrypoint
* simple env example
* make takedown ids opaque identifiers in the pds
* use pds routes for api tests
* update pds dockerfile with volume and correct port env var
* add a couple env vars to example
* add comments to env example
* @atproto/pds 0.2.0-beta.0
* @atproto/aws 0.0.1-beta.0
* appview did
* @atproto/aws 0.0.1
* enable logs by default
* update env example
* bugfixing sandbox issues
* consistency in pds env var name for appview url
* log on pds start and stop, configure version at runtime
* @atproto/pds 0.2.0-beta.1
* fix semver matching for pds beta version
* v0.2.0-beta.2
* default invites to being not required
* fix flaky test
* limit db connections in tests
* publish 0.2.0-beta.d3
* fix invite required parsing
* @atproto/pds 0.2.0-beta.5
* Proxy getPopularFeedGenerators on simplified pds (#1222)
proxy getPopularFeedGenerators on pds
Co-authored-by: dholms <dtholmgren@gmail.com>
* tidy migrations
* fix service entry
* bump version
* change auth order
* bump version
* bump version
* add upgradeRepoVersion & fallback url for cdn
* bump version
* merging
* merge pds
* building dev-env
* merging tests
* merge service entry
* test fixing
* tidy
* fix admin search
* tidy
* tidy
* add snap for getListFeed
* add backup nameserver cfg
* tidy + pr feedback
* tidy
* tidy env
* bit more
* re-add dotenv to root package.json
* fix dep
* build branch
* fix tests
* Refactor tests to make better use of dev-env (#1690)
* refactor pds tests to use dev env
* refactor bsky tests
* fix pds test
* tidy bsky tests
* build pds correctly
* fix entry point
* default logging to false (for now)
* format service entry
* Switch takedown ids back to ints on pds distribution (#1694)
* switch takedown ids back to ints, consistent with live pds
* tidy/fix migration
* update migration for sqlite
* export moderation action reversal
* takedown tests
* dont build branch
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* add compression
* compress carfiles
* update package version
* add tests
* one more test
* remove compressible dep
* add gzip to appview
* fix package.json
* wip
* first pass done
* chnage around how channels function
* bugfixing
* fix up migration & tests
* fix dev-env
* add prep migration for stream fixing
* application code changes to use new repo_event table
* dont send evts
* add outgoing_repo_seq table + sqlite migrations
* patch up down migration
* patching up a couple things
* update test
* patchup
* re-enable subscription route
* add outgoingSeq column
* updating
* accidental add
* temporarily disable stream
* resolve merge conflict
* add migration
* migration for seq
* fix test
* id -> seq
* get schema lock ids
* tidy
* fix server test
* fix up
* use bsky uniqueLockId approach
* tidy
* gh actions build
* build branch
* build branch
* tweak test
* rm constraint rename
* small bugfix
* check for seq not null
* fix up issues
* one more small change
* tidy
* no build branches
* Scaffold xrpc-stream
* Write and test stream frames
* Write and test stream server
* Test bad stream frame parsing cases
* Proof of concept streaming on xrpc-server
* Test bad streaming endpoint
* Make frame stream to fix buffering frame bytes together
* reorg
* some lex-cli things
* better codegen for subscriptions
* scaffolding repo subscriptions
* wip
* remove repo ops
* setup notify/listen for db
* end pool to fix hanging test
* small comment in test
* basic sequencer
* some refactoring
* switch to event emitter
* reconnect on listener error
* rename notifyClient
* remove payload on channels
* pr feedback
* subscribeRepo outbox
* some cleanup
* wip
* wip
* bugfixin
* only send msgs after tx is committed
* better handle event-emitter -> generator
* max buffer size
* cleanup
* Sync-up xrpc subscriptions with current iteration of the spec
* Allow missing encoding for subscriptions
* track sequencedAt & eventType as well
* Resolve streaming type codes, streaming tests
* Fix interface for hooking into http server for streaming, fix stream routing
* Minor reorg how streaming is enabled in xrpc-server
* Server codegen for xrpc subscriptions, supporting types in xrpc-server
* fix up buffer overloading
* Rename data frame to message frame, rename subscription lex output to message
* Move body param first for streaming frames
* Tidy
* clean up evt types
* buff up tests
* missed merge conflict
* new schema
* blobs on subscriptions
* rm genned client subscription methods
* backfill limits
* testing subscription route & quick outbox bugfix
* fix up migration
* cascade on delete
* comments & naming
* fix dev env
* delete seqs on account deletion
* tidy
* fixing things up with db notify system for schemas
* fix duplicates in outbox
* tidy
* Break out of stream handler when socket closes
* fixing up some timing issues
* tidy
* terminate ws after using
* bump up timer on async reader
* fixing up NOTIFY in txns
* pr feedback
* pr bugfixes
* make order asc explicit
* bringing tests up to speed w atpagent
* bump up max listeners on sequencer
* increase timeouts for reading from generators
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* Refactor @atproto/api to use the simplified AtpAgent API
* xrpc package: Export the defaultFetchHandler to reuse in api
* api package: Use the defaultFetchHandler defined in xrpc
* Update all usages of the api for the new AtpAgent
* Clear promise on thrown codepath
* Avoid updating the atpagent session until ready to return
* Model mod actions for actor takedowns
* Add administration lex schemas for moderation actions
* Initial implementation of takedown mod action and reversal
* Refactor admin actions into a service
* Check for account takedown when creating new session or refreshing session
* Sweep usage of getUser() and getUserByEmail() for takedowns
* Begin sweep for account takedowns
* Tidy takedown checks
* Handle post author takedowns
* Fix tests and migrations for mod takedown
* Fix xrpc-server delayed read of blob payloads
* Test takedown mod action, misc fixes
* Remodel lex schemas for admin moderation actions
* Remove delcaration cid from actor takedowns, lex schema tidy
* Add notes
* Move moderation methods to com.atproto, based around repo rather than actor
* Moderation reports of repos and records
* Implement moderation report resolution method
* Tidy
* Place moderation-init migration after mute-init
* Allow services to reference each other
* Model record takedowns in db
* Lex schemas for record takedowns
* Service/methods for record takedowns, fix report view subject
* Apply record takedown to posts in feeds
* Test record takedown effects, fix reversal
* Support server moderation flag and acknowledge actions, fix action/report resolution for records
* Track repo takedowns on repo_root rather than did_handle
* Move repo/record reporting under com.atproto.report
* Colocate all moderation functionality into moderation service, combine from admin and repo services
* Tidy
* Tidy
* Unify mod report and action methods with new repo/record ref schemas
* introducing appcontext & pds class
* revamp rest of server + auth
* revamp tests
* add to dev-env
* fix api test
* add start method to pds
* hook up terminator before
* PLC AppContext (#425)
* convert plc server to app context
* add start method to plc
* hooke up terminator on server setup
* fix test
* cleanup & pr feedback
* fix up one more test
* Add auth to xrpc-server, refactor to use per-route middleware stack
* Add xrpc-server error middleware, tidy
* Tests in xrpc for for catchall errors and auth
* Tidy
* Update lex-cli for new xrpc-server auth interface
* Update pds lexicon w/ codegen for xrpc auth
* Use xrpc-server auth in pds
* Update tests for newly-required auth
* Sketching out max payload size in xrpc-server
* Fixes and testing for external payload parsing
* Apply max json payload limit to pds and plc
* Move parsing middleware into xrpc-server w/ options
* Add plc healthcheck, unify plc server interface with pds
* Add pds healthcheck
* Make plc test util file consistent with other packages
* Fix types in dev-env
* Type fixes
* Tidy
* Rename healthcheck routes, place pds's under /xrpc
