* Improve error message when using invalid client_id during code exchange
* Extract SPA example OAuth client in own package
* wip
* remove dependency on get-port
* Properly configure jest to only transpile "get-port" from node_modules
https://jestjs.io/docs/configuration#transformignorepatterns-arraystring
* Use dynamically assigned port number during tests
* use puppeteer to run tests
* remove login input "id" attribute
* code style
* add missing declaration
* tidy
* headless
* remove get-port dependency
* fix tests/proxied/admin.test.ts
* fix tests
* Allow unsecure oauth providers through configuration
* transpile "lande" during ozone tests
* Cache Puppeteer browser binaries
* Use puppeteer cache during all workflow steps
* remove use of set-output
* use get-port in xrpc-server tests
* Renamed to allowHttp
* tidy
* tidy
* feat(api): support creation of oauth based AtpAgents
* oauth: misc fixes for confidential clients
* fix(xprc): remove ReadableStream.from polyfill
* OAuth docs tweaks (#2679)
* OAuth: clarification about client_name being shown
* OAuth: re-write handle resolution privacy concern
* avoid relying on ReadableStream.from in xrpc-server tests
* feat(oauth-types): expose "ALLOW_UNSECURE_ORIGINS" constant
* feat(handle-resolver): expose "AtprotoIdentityDidMethods" type
* fix(oauth-client): ensure that the oauth metadata document contains client_id_metadata_document_supported
* fix(oauth-types): prevent unknown query string in loopback client id
* fix(identity-resolver): check that handle is in did doc's "alsoKnownAs"
* feat(oauth-client:oauth-resolver): allow logging in using either the PDS URL or Entryway URL
* fix(oauth-client): return better error in case of invalid "oauth-protected-resource" status code
* refactor(did): group atproto specific checks in own
* feat(api): relax typing of "appLabelers" and "labelers" AtpClient properties
* allow any did as labeller (for tests mainly)
* fix(api): allow to override "atproto-proxy" on a per-request basis
* remove release candidate versions from changelog
* update changeset for api and xrpc packages
* Add missing changeset
* revert RC versions
* Proper wording in OAUTH.md api example
* remove "pre" changeset file
* xrpc: restore original behavior of setHEader and unsetHeader
* docs: add comment for XrpcClient 's constructor arg
* feat(api): expose "schemas" publicly
* feat(api): allow customizing the whatwg fetch function of the AtpAgent
* docs(api): improve migration docs
* docs: change reference to BskyAgent to AtpAgent
* docs: mention the breaking change regarding setSessionPersistHandler
* fix(api): better split AtpClient concerns
* fix(xrpc): remove unused import
* refactor(api): simplify class hierarchu by removeing AtpClient
* fix(api): mock proper method for facets detection
* restore ability to restore session asynchronously
* feat(api): allow instantiating Agent with same argument as super class
* docs(api): properly extend Agent class
* style(xrpc): var name
* docs(api): remove "async" to header getter
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Hailey <me@haileyok.com>
* fix(xrpc-server): add exception for HTTP clients that set a content-type when there is no body
* feat(xrpc-server): allow empty body on endpoints not expecting any input
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* fix(xrpc-server): properly test empty body requests
* tidy
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* chore(deps): update zod
* chore(deps): update pino to match entryway version
* chore(tsconfig): remove truncation of types through noErrorTruncation
* add support for DPoP token type when logging
* fix(bsky): JSON.parse does not return value of type JSON
* fix(pds): add res property to ReqCtx
* fix(pds): properly type getPreferences return value
* chore(tsconfig): disable noFallthroughCasesInSwitch
* refactor(pds): move tracer config in own file
* feat(dev-env): start with "pnpm dev"
* feat(oauth): add oauth provider & client libs
* feat(pds): add oauth provider
* chore: changeset
* feat: various fixes and improvements
* chore(deps): update better-sqlite3 to version 10.0.0 for node 22 compatibility
* chore(deps): drop unused tslib
* fix(did): normalize service IDs before looking for duplicates
* fix(did): avoid minor type casting
* fix(did): improve argument validation
* fix(fetch): explicit use of negation around number comparison
* fix(oauth-provider): improve argument validation
* feat(did): add ATPROTO specific "isAtprotoDidWeb" method
* feat(rollup-plugin-bundle-manifest): add readme
* feat(lint): add eqeqeq rule (only allow == and != with null)
* fix(oauth-client-browser): typo in gitignore
* fix(oauth-provider): properly name error class file
* fix(oauth-provider): remove un-necessary useMemo
* fix(did-resolver): properly build did:web document url
* fix(did-resolver): remove unused types
* fix(fetch): remove unused utils
* fix(pds): remove unused script and dependency
* fix(oauth-provider): simplify isSubPath util
* fix(oauth-provider): add InvalidRedirectUriError static constructor
* fix(jwk): improve JWT validation to provide better error messages and distinguish between signed and unsigned tokens
* fix(pds): use "debug" log level for fetch method
* fix(pds): allow access tokens to contain an unknown "typ" claim (with the exception of "dpop+jwt")
* fix(jwk): remove un-necessary code
* fix(pds): account for whitespace chars when checking JSON
* fix(pds): remove oauth specific config
* fix(pds): run all write queries through transaction or executeWithRetry
fix(pds): remove outdated comments
fix(pds): rename used_refresh_token columns & added primary key
fix(pds): run cleanup task through backgroundQueue
fix(pds): add device.id foreign key to device_account
fix(pds): add comment on cleanup of used_refresh_token
fix(pds): add primary key on device_account
* fix(oauth-provider:time): simplify constantTime util
* fix(pds): rename disableSsrf into disableSsrfProtection
* fix(oauth-client-react-native): remove incomplete package
* refactor(pds): remove status & active from ActorAccount
* fix(pds): invalidate all oauth tokens on takedown
* fix(oauth-provider): enforce token expiry
* fix(pds): properly support deactivated accounts
* perf(pds:db): allow transaction function to be sync
* refactor(psq:account-manager): expose only query builders & data transformations utils from helpers
* fix(oauth-provider): imports from self
* fix(ci): add nested packages to build artifacts
* style(fetch): rename TODO into @TODO
* style(rollup-plugin-bundle-manifest): remove "TODO" from comment
* style(oauth-client): rename TODO into @TODO
* style(oauth-provider): rename TODO into @TODO
* refactor(oauth-client): remove "OAuth" prefix from types
* fix(oauth-client-browser): better type SessionListener
* style(oauth): rename TODO into @TODO
* fix(oauth-provider): enforce provider max session age
* fix(oauth-provider): check authentication parameters against all client metadata
* fix(api): tests
* fix(pds): remove .js from imports for tests
* fix(pds): change account status to match tests
* chore(deps): make all packages depend on the same zod version
* fix(common-web): remove un-necessary binding of Checkable to "zod"
* refactor(jwk): infer jwt schema from refinement definition
* fix(handle-resolver): allow resolution errors to propagate
docs(handle-resolver): better handling of DNS resolution errors
fix(handle-resolver): properly handle DOH responses
* fix(did): service endpoint arrays must contain "one or more" element
* refactor(pipe): simplify implementation
* fix(pds): add missing DB indexes
* feat(oauth): Resolve Authorization Server URI through Protected Resource Metadata
* style:(oauth-client): import order
* docs(oauth-provider:redirect-uri): add reference url
* feat(oauth): implement "OAuth Client ID Metadata Document" from draft-parecki-oauth-client-id-metadata-document-latest internet draft
* feat(oauth-client): backport changes from feat-oauth-client
* docs(simple-store): improve comments
* feat(lexicons): add iterable capabilities
* fix(pds): type error in dev mode
* feat(oauth-provider): improved error reporting
* fix(oauth-types): allow insecure issuer during tests
* fix(xrpc-server): allow upload of empty files
* fix: lint
* feat(fetch): keep request reference in errors
feat(fetch): utilities improvements
* fix(pds): allow more than one session token per user
* feat(ozone): improve env validation error messages
* fix(oauth-client): account for DPoP when checking for invalid_token errors
* fixup! feat(fetch): keep request reference in errors feat(fetch): utilities improvements
* fixup! feat(fetch): keep request reference in errors feat(fetch): utilities improvements
* fix(oauth): various validation fixes
feat(oauth): share client_id validation and parsing utilities between client & provider
* feat(dev-env): fix ozone port number
* fix(fetch-node): prevent fetch against invalid domain names
* fix(oauth-provider): add typings for psl dep
* feat(jwk): make type def compatible with TS 4.x
* fix(oauth): fixed various spec compliance
fix(oauth): return "sub" in refresh token response
fix(oauth): limit token validity for third party clients
fix(oauth): hide client image when not trusted
* fix(oauth): lint
* pds: switch changeset to patch, no breaking changes
* changeset and config for new oauth deps
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* nsid: reduce 'name' length from 128 chars to 63 chars
* nsid: remove 'nsid-ns' (glob pattern) from main NSID syntax validation
This special variant syntax should be handled separately. Eg, an
"nsid-ns" should not be accepted as an NSID in Lexicons looking for a
regular NSID.
* nsid: update syntax check to match domain rules
This is to be closer to actual underlying specification needs. For
example, domains are allowed to start with a digit (like 4chan.org), and
can't start *or* end with a hyphen.
Restricts the "name" part further, to be alphabetic only.
Note that starts-with-digit domains could break some lang
auto-generation variable name rules. The docs/spec strongly recommend
against such domains for use with NSID, but it seems incorrect to
restrict at the standard/lexicon level without a clear pan-language
consistent set of rules.
* did-resolver: disallow did:web with path segments
* identifier: reduce max length of DID from 8 KB to 2 KB
* identifier: add DID test with escaped ':' in identifier
* did-resolver: have prettier run on tests/
* identifier: add additional punycode test cases
* identifiers: small tweaks from review
- type in did:web comment
- include actual max DID string length in error message
* xrpc-server tests: remove digits from NSIDs
* identity: fix import and function errors
* xrpc-server tests: remove digits from NSIDs
* xrpc-server: include tests/ in prettier:fix
* xrpc-server: lint fix
* remove return in test
* couple of fixups in other pacakges
* Add dummy checks to declaration and follow app migrations, remove paranoid join
* update db nsid migration
* Ensure there are writes in follow app migration
* Add dumy check to votes-to-likes app migration, tidy
* Ensure there are writes in vote-to-like app migration
* update migration name
* pr feedback
* count utf8 & grapheme length
* add maxUtf8
* siwtch max semantics
* plural
* update post schema
* added bytes & cid refs
* add ipld<>json
* fixin up a could tings
* Add app.bsky.richtext.facet, replace post entities with facets
* plural actors
* wip
* Setup backlinks table on pds
* wip
* send & recieve cids/bytes with xrpc
* Track backlinks when indexing records on pds
* handle ipld vals in xrpc server
* added cids & bytes to codegen
* In createRecord, add deletions to avoid duplicate likes/follows/reposts
* Tests and fixes for prevention of dupe follows, likes, reposts
* Backlink migration tidy
* cleanup dag json parser
* Fix dupe backlink inserts
* Tidy
* blob refs + codegen
* Make profile displayName optional
* Test view and updateProfile for empty display name
* working into pds
* Make aggregate counts optional on post and profile views
* Make viewer state optional on post view for consistency
* Remove deprecated myState field on profile view
* Tidy repo method descriptions
* tests & types & fixes
* Implementation and tests for putRecord
* Remove updateProfile method
* Update repo service so that head can be taken for update externally
* Lex updates for compare-and-swap records/commits
* Add error to lex for bad repo compare-and-swaps
* Improve update-at-head thru repo service
* common package
* Implement and test compare-and-swaps on repo write methods
* Use lex discriminator for applyWrites
* Remove post entity/facet index
* Update lex descriptions to clarify repo write semantics
* Make deleteRecord idempotent w/ tests
* cleanup
* fix things up
* adding more formats
* tests
* updating schema
* Only generate tid rkeys on pds, support literal rkeys on client
* Add backlink indexes
* Update format of post embed views, fix external uri validation
* fixing up tests
* Include embeds on record embeds
* cleanup
* Notify users when they are quoted
* Remove determineRkey indirection
* fix api tests
* support concatenated cbor
* integrating to server
* re-enable tests
* fix up tests
* Thread compare-and-swaps down into repo service rather than use pinned storage
* Tidy
* Update packages/common/tests/ipld-multi.test.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* Update packages/lexicon/src/validators/formats.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* pr feedback
* pr feedback
* Add postgres-specific migration path for missing profile display names
* Tidy/clarify deep embeds
* Tidy
* rm unused escape
* decrease crud race count
* update subscribeRepos lexicon
* Fix applyWrite lexicon re: collection fields
* sign post event type
* update cids & bytes json encoding
* update lex blob & cid-link types
* updated codegen & pds
* number -> float
* missed a couple
* remove old image constraints
* pr feedback + descripts
* no hardcoded port numbers
* remove separate tooLarge evt
* fix dumb build error
* fixin gup lex + xrpc server
* better parsing of message types
* dont mutate body in subscription
* bugfix in subscription
* rm commented out code
* init feature branch
* undo
* Remove old lexicons
* Remove creator from profile view
* wip
* rework seqs
* fixed up tests
* bug fixing
* sequence handles & notify in dbTxn
* tidy
* update lex to include times
* test syncing handle changes
* one more fix
* handle too big evts
* dont thread sequencer through everything
* Split common into server vs web-friendly versions
* Make lexicon, identifier web-safe using common-web
* Switch api package to be a browser build, fix identifier package for browser bundling
* Fix pds and repo for lexicon package changes, tidy
* Make common-web a browser build, tidy
* fixing up deps
* fix up test
* turn off caching in actions
* Standardize repo write interfaces around repo input
* Update repo write endpoints for repo input field
* Remove scene follows during app migration
* API package updates (#712)
* Add bsky agent and various sugars to the api package
* Add richtext library to api package
* Update richtext to use facets and deprecate entities
* Update richtext to use utf8 indices
* Richtext converts deprecated entity indices from utf16 locations to utf8 locations
* Add note about encodings in the lexicon
* Add RichText facet detection
* Remove dead code
* Add deprecation notices to lexicons
* Usability improvements to RichText
* Update the api package readme
* Add RichText#detectFacetsWithoutResolution
* Add upsertProfile to bsky-agent
* Update packages/pds/src/api/com/atproto/repo/applyWrites.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* pr feedback
* fix flaky timing streaming tests
* simplify emptyPromise
* fixed up open handles
* fix missed repo syntax
* fix error in test from fkey constraint
* fix another api agent bug
* Embed consistency, add complex record embed
* Tidy embed lex descriptions
* rename pg schemas
* use swc for jest
* fix up deps
* cleanup
* Update pds indexing, views, tests for complex record embeds
* fixing up profile view semantics
* wip
* update snaps
* Rename embed.complexRecord to embed.recordWithMedia
* Tidy aroud record w/ media embeds
* Add grapheme utilities to api RichText (#720)
Co-authored-by: dholms <dtholmgren@gmail.com>
* Fix: app.bsky.feed.getPostThread#... to app.bsky.feed.defs#... (#726)
* Update bskyagent to use repo param
* Minor typing fix
* Add exports to api package: blobref & lex/json converters (#727)
* Add exports to api package: BlobRef & lex/json converters
* Add an example react-native fetch handler
* Switch all lingering references of recordRef to strongRef
* Update lexicon for richtext facets to have multiple features, byte slice rather than text slice
* Implement multi-feature richtext facets on pds
* Update api package to use updated richtext facets
* Minor fixes to admin repo/record views
* Fix app migration exports, remove old app migration
* Fix: sort richtext facets so they can render correctly
* Disable app migration dummy checks that don't work on live deploy
* Optimize lex de/serialization using simple checks
* Tidy comment typos
* App migration to cleanup notifications for likes, follows, old scene notifs
* Fix notification reason for change from vote to like
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
* pds: fix logging of Bearer auth subject (DID)
* xrpc-server: handle missing and mangled Content-Type better
Eg, for blob uploads.
Also adds a test for incorrect mimetype supplied for a regular endpoint
(one not allowing "*/*").
* skip failing test
---------
Co-authored-by: dholms <dtholmgren@gmail.com>
* Support compression in requests to xrpc-server
* Check payload streaming payload sizes in xrpc-server
* Tidy tests
* Move MaxSizeChecker into common stream utils
* Add auth to xrpc-server, refactor to use per-route middleware stack
* Add xrpc-server error middleware, tidy
* Tests in xrpc for for catchall errors and auth
* Tidy
* Update lex-cli for new xrpc-server auth interface
* Update pds lexicon w/ codegen for xrpc auth
* Use xrpc-server auth in pds
* Update tests for newly-required auth
* Implement types and validation for new NIH lexicon DSL
* Update xprc and xrpc-server to use new NIH lexicon DSL
* Simplification: dont allow lexicon definitions to be refs or ref arrays
* Add 'unknown' primitive type to NIH lexicon
* Implement client codegen for NIH Lexicon
* Implement server codegen for NIH Lexicon
* Add the 'datetime' primitive type to the NIH Lexicon
* Rework lexicons to use new NIH Lexicon DSL
* Readd tokens to code generation
* Update all code to use new NIH Lexicon (all tests passing!)
* Add imports from external files to lex-cli codegen
* Do not allow records, queries, or procedures outside of the main def
* Update packages/lex-cli/src/util.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* Linter pass
* Add knownValue output to codegen
* Add comments to codegen
* Use enum and const in codegen
* Rework lexicon 'properties' structure
* Treat falsy query params as undefined
Co-authored-by: devin ivy <devinivy@gmail.com>
* Remove now-redundant param presence validation
Co-authored-by: devin ivy <devinivy@gmail.com>
* Sketching out max payload size in xrpc-server
* Fixes and testing for external payload parsing
* Apply max json payload limit to pds and plc
* Move parsing middleware into xrpc-server w/ options
* Model procedure for setting a vote on a post
* Implement app.bsky.feed.setVote() method
* Support json schema subset for parameters in lexicon and lex-cli
* Update lexicons w/ json schema parameters
* Use enum in getVotes parameters
* Support json schema subset for parameters in xrpc-server, ajv validation
* Include parameter enums in generated markdown
* Tidy
* Unify xrpc parameter md docs with inputs and outputs: show ts interfaces
* Tidy
* Rename ADX to ATP
* Remove old docs (now in atproto website repo)
* Update readme
* Update README.md
Co-authored-by: devin ivy <devinivy@gmail.com>
* A couple of @ -> AT
Co-authored-by: devin ivy <devinivy@gmail.com>
* Add xrpc-cli package; implement 'new' and 'gen-md' commands
* Add gen-ts-obj to xrpc-cli
* Add gen-api command to xrpc-cli
* Add gen-server to xrpc-cli
* Update codegen from xrpc-cli to run prettier and include a banner comment
* Use 'void' instead of 'undefined' for return type
* Skip invalid schemas when generating code
* Rewrite schemas package to use new simplified schema
* Output type definitions in nested folders
* Add record schema code-generation
* Change xrcp client call() signature to be easier to use
* Update client codegen to match new xrpc call signature
* Fixes and improvements to generated record APIs
* Update generated api to use per-service instances
* Add xrpc-cli readme
* Rename fedrpc to xrpc; implement xrpc and xrpc-server packages
* Add json-schema validation to xrpc
* Add xrpc and xrpc-server to base tsconfig.json
* Fix type signature
