122 Commits

Author SHA1 Message Date
github-actions[bot]
e216e87859
Version packages (#4167)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-09-09 12:29:10 +02:00
Matthieu Sieben
055a413fba
InternalServerError when creating records (#4169)
* Various perf fixes

* add transaction assertions

* tidy

* changeset

* tidy

* Update packages/aws/src/s3.ts

* tidy

* Apply suggestions from code review

* tidy

* Update .changeset/chilled-shirts-ring.md

Co-authored-by: devin ivy <devinivy@gmail.com>

* Update .changeset/stale-rocks-press.md

* Update packages/pds/src/actor-store/blob/transactor.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* build

* revert blob upload logic

* tidy

* use `uploadTimeoutMs` as default for `requestTimeoutMs`

* review coments

* chngeset

* tidy

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
Co-authored-by: David Buchanan <david@blueskyweb.xyz>
2025-09-08 18:40:56 +02:00
github-actions[bot]
920f895807
Version packages (#4152)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-08-29 12:54:51 +02:00
github-actions[bot]
768e81b232
Version packages (#4126)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-08-27 13:36:05 -04:00
github-actions[bot]
5188ef3b59
Version packages (#4116)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-08-20 21:48:51 +02:00
devin ivy
331a356ce2
Lexicon resolver package (#4069)
* lexicon: doc validation compatibility with published lexicons

* lexicon-resolver: setup new package

* lexicon-resolver: implement record resolution

* lexicon-resolver: implement lexicon resolution

* lexicon-resolver: test record resolution

* repo: add option to verify CIDs found in CARs. tidy.

* lexicon-resolution: verify CIDs in proof CAR

* lexicon-resolution: tests and fixes

* tidy

* lexicon-resolution: add entrypoint

* lexicon-resolver: tidy errors

* lexicon-resolver: readme

* lexicon-resolver: changeset

* prettier

* eslint

* tidy

* tidy

* tidy

* enable CID-to-content verification within CARs by default

* lexicon-resolver: tidy types, application of defaults, gitattributes

* lexicon-resolver: add interface and builder fn for lexicon and record resolvers

* lexicon-resolver: update readme

* tidy

* lexicon-resolver: cover error cases in record resolution

---------

Co-authored-by: Matthieu Sieben <matthieu.sieben@gmail.com>
2025-08-17 22:45:51 -04:00
github-actions[bot]
f27ae66432
Version packages (#4024)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-07-11 20:35:18 +02:00
github-actions[bot]
b70f62c6b9
Version packages (#3988)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-06-24 06:17:02 -07:00
bnewbold
a8dee6af33
repo: MST should allow tilde in keys (#3981)
* repo: MST should allow tilde in keys

* add changeset
2025-06-22 17:43:06 -07:00
github-actions[bot]
9f9a08648b
Version packages (#3975)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-06-19 20:43:13 +02:00
Paul Frazee
5fccbd2a14
Replace slice() with subarray() in car file parsing (#3971)
* Replace slice() with subarray() in car file parsing

* changeset

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2025-06-18 20:39:05 -07:00
github-actions[bot]
b4cb1ebac9
Version packages (#3959)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-06-17 15:50:58 +02:00
bnewbold
8bd45e2f89
bump MST key length from 256 to 1024 chars (#3956)
* bump MST key length from 256 to 1024 chars

* update MST key test

* add a changeset
2025-06-14 11:39:56 -07:00
github-actions[bot]
b36014dac5
Version packages (#3803)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-05-05 17:44:12 -03:00
github-actions[bot]
6bc8355c40
Version packages (#3710)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-02 18:38:47 -05:00
Daniel Holmgren
4db923ca1c
Rewrite CAR implementation (#3672)
* use hand-written car writer

* tidy

* initial impl

* fix bug & compiler errors

* remove dependency

* refactor & tidy

* del old util test

* simple test fixtures

* changesets

* propagate termination

* pr feedback

* support async & sync iterables

* add dump methohd to iterator

* build branch

* fix import

* dont build branch
2025-04-02 16:57:32 -05:00
github-actions[bot]
a26813da21
Version packages (#3703)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-04-01 15:30:27 -05:00
Daniel Holmgren
bdbd3c3e3f
Sequencer recovery (#2519)
* wip

* flesh out recoverer

* basic tests + handle uncreated repos

* add key recovery when keys are lost

* schemas

* tidy recoverer

* tidy & comment test

* work into script framework

* use account evt instead of tombstone

* add flag for updating keys

* add log

* rotate keys script

* pr feedback

* build branch

* use exact commit data from sequencer to restore

* fixing up key rotation script

* add onIdle

* build branch

* track blobs

* fix types

* fix blob tracking

* patches

* publish identity script

* fix build err

* wip

* build err

* wip

* recovery db + fix some errors

* refactor & add repair script

* dont run script

* fix test

* tidy scripts

* tidy

* change publish identity recoveyr method to rotate keys

* tidy

* pr feedback

* simple readme

* changesets
2025-03-31 17:02:11 -05:00
github-actions[bot]
c777ba6d68
Version packages (#3631)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2025-03-20 15:21:15 +01:00
github-actions[bot]
f46554bcb8
Version packages (#3591)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-03-07 11:03:00 -05:00
github-actions[bot]
95299e0f62
Version packages (#3566)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-02-24 14:29:37 +01:00
Daniel Holmgren
7e3678c089
Send prevs on firehose (#3449)
* schema

* reset rate limit codegen

* codegen

* send prev cids on firehose

* fix test

* fix some test compiler errors & add experimental note

* fix linting

* build branch

* add prevData to commit event

* fix cbor undefined err

* add sibling proofs to relevant blocks

* bump depth of obj in test

* fix bug on right sibling proof & add some tests

* another test

* refactor proof construction

* more tests

* factor into fixtures

* fix styles in json

* lint: import ordering

* pr feedback

* add invertible op test

* remove prev from outgoing events

* return to original proof construction

* dont build branch

* changeset
2025-02-21 15:01:08 -06:00
github-actions[bot]
03351a5818
Version packages (#3529)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-02-13 15:27:09 +01:00
Matthieu Sieben
c53d943c8b
Improve codegen typings (#2999)
* Make codegen types stricter
* Add .js file extension to import statements in generated code
* Fixes a bug that would clear interests prefs when updating hidden posts prefs.
2025-02-13 15:21:00 +01:00
github-actions[bot]
799dd925e9
Version packages (#3493)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-02-05 17:50:10 -06:00
Matthieu Sieben
61dc0d60e1
Add linting rule to sort imports (#3220)
* Add linting rule to sort imports

* remove spacing between import groups

* changeset

* changeset

* prettier config fine tuning

* forbid use of deprecated imports

* tidy
2025-02-05 15:06:58 +01:00
github-actions[bot]
1c195a3845
Version packages (#3442)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-01-31 18:01:32 -06:00
github-actions[bot]
5b6e0611d6
Version packages (#3336)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-01-07 12:06:16 -06:00
github-actions[bot]
7aecc57dbb
Version packages (#3331)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-01-07 09:29:57 -05:00
github-actions[bot]
51b0c48ce7
Version packages (#3188)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-12-11 14:11:08 -06:00
Daniel Holmgren
c9848edaf0
Fix missing delete blocks (#3033)
* send relevant blocks on commit

* use relevantBlocks in pds

* test

* fix compiler errors

* send both new & relevant blocks

* build branch

* changsets

* no build branch
2024-12-05 18:16:10 -06:00
github-actions[bot]
3a5fc92a74
Version packages (#2962)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-11-13 17:00:25 -06:00
github-actions[bot]
b398276b36
Version packages (#2932)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-11-08 10:22:43 +01:00
Matthieu Sieben
9d40ccbb69
Various OAuth related fixes (#2871)
* wip

* tidy

* tidy

* tidy

* Update packages/oauth/oauth-client/src/session-getter.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* fix combineSignals

* tidy

* tidy

* improve typing of atprotoScopeSchema

* stronger typings

* tidy

* ci

* Fix cors error

* downgrade ioredis dependency

* fix ioredis version

* tidy

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2024-10-18 20:23:33 +02:00
Matthieu Sieben
fabc8a9381
Update typescript to version 5.6.2 (#2863) 2024-10-11 14:05:53 +02:00
github-actions[bot]
a611a5fe56
Version packages (#2846)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-01 20:13:26 +02:00
github-actions[bot]
6593fdc3f4
Version packages (#2812)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-27 13:04:34 -05:00
github-actions[bot]
85c85350d1
Version packages (#2791)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-11 18:30:47 -05:00
github-actions[bot]
0a37a3cc56
Version packages (#2788)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-04 20:22:22 -05:00
Daniel Holmgren
b15dec2f4f
Atproto sync package (#2752)
* first pass/port

* reworking

* authenticated commit parsing

* authenticate identity evts

* some testing

* tidy & add firehose to queue

* error handling

* fix test

* refactor sync queue + some tests

* fix race in sync queue

* rm firehose from syncqueue

* add tests for queue utils

* README

* lint readme

* filter before parsing

* pr feedback

* small fix

* changesets

* fix type

* Rework dataplane subscription (#2766)

* working sync package into appview subscription

* add restart method to subscription for tests

* fix another test

* tidy subscription utils/files

* remove dupe property

* tidy after merge

* fix start cursor on subscription

* tweak process full subscription logic

* fixes
2024-09-04 20:18:16 -05:00
github-actions[bot]
a1d8c77edd
Version packages (#2738)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-27 14:03:39 -04:00
github-actions[bot]
3940733bf0
Version packages (#2706)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-12 16:11:19 -04:00
github-actions[bot]
f2f8de63b3
Version packages (#2639)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-07-12 17:34:56 +02:00
Matthieu Sieben
a8d6c11235
🚧 OAuth2 - Authorization Server (#2482)
* chore(deps): update zod

* chore(deps): update pino to match entryway version

* chore(tsconfig): remove truncation of types through noErrorTruncation

* add support for DPoP token type when logging

* fix(bsky): JSON.parse does not return value of type JSON

* fix(pds): add res property to ReqCtx

* fix(pds): properly type getPreferences return value

* chore(tsconfig): disable noFallthroughCasesInSwitch

* refactor(pds): move tracer config in own file

* feat(dev-env): start with "pnpm dev"

* feat(oauth): add oauth provider & client libs

* feat(pds): add oauth provider

* chore: changeset

* feat: various fixes and improvements

* chore(deps): update better-sqlite3 to version 10.0.0 for node 22 compatibility

* chore(deps): drop unused tslib

* fix(did): normalize service IDs before looking for duplicates

* fix(did): avoid minor type casting

* fix(did): improve argument validation

* fix(fetch): explicit use of negation around number comparison

* fix(oauth-provider): improve argument validation

* feat(did): add ATPROTO specific "isAtprotoDidWeb" method

* feat(rollup-plugin-bundle-manifest): add readme

* feat(lint): add eqeqeq rule (only allow == and != with null)

* fix(oauth-client-browser): typo in gitignore

* fix(oauth-provider): properly name error class file

* fix(oauth-provider): remove un-necessary useMemo

* fix(did-resolver): properly build did:web document url

* fix(did-resolver): remove unused types

* fix(fetch): remove unused utils

* fix(pds): remove unused script and dependency

* fix(oauth-provider): simplify isSubPath util

* fix(oauth-provider): add InvalidRedirectUriError static constructor

* fix(jwk): improve JWT validation to provide better error messages and distinguish between signed and unsigned tokens

* fix(pds): use "debug" log level for fetch method

* fix(pds): allow access tokens to contain an unknown "typ" claim (with the exception of "dpop+jwt")

* fix(jwk): remove un-necessary code

* fix(pds): account for whitespace chars when checking JSON

* fix(pds): remove oauth specific config

* fix(pds): run all write queries through transaction or executeWithRetry
fix(pds): remove outdated comments
fix(pds): rename used_refresh_token columns & added primary key
fix(pds): run cleanup task through backgroundQueue
fix(pds): add device.id foreign key to device_account
fix(pds): add comment on cleanup of used_refresh_token
fix(pds): add primary key on device_account

* fix(oauth-provider:time): simplify constantTime util

* fix(pds): rename disableSsrf into disableSsrfProtection

* fix(oauth-client-react-native): remove incomplete package

* refactor(pds): remove status & active from ActorAccount

* fix(pds): invalidate all oauth tokens on takedown

* fix(oauth-provider): enforce token expiry

* fix(pds): properly support deactivated accounts

* perf(pds:db): allow transaction function to be sync

* refactor(psq:account-manager): expose only query builders & data transformations utils from helpers

* fix(oauth-provider): imports from self

* fix(ci): add nested packages to build artifacts

* style(fetch): rename TODO into @TODO

* style(rollup-plugin-bundle-manifest): remove "TODO" from comment

* style(oauth-client): rename TODO into @TODO

* style(oauth-provider): rename TODO into @TODO

* refactor(oauth-client): remove "OAuth" prefix from types

* fix(oauth-client-browser): better type SessionListener

* style(oauth): rename TODO into @TODO

* fix(oauth-provider): enforce provider max session age

* fix(oauth-provider): check authentication parameters against all client metadata

* fix(api): tests

* fix(pds): remove .js from imports for tests

* fix(pds): change account status to match tests

* chore(deps): make all packages depend on the same zod version

* fix(common-web): remove un-necessary binding of Checkable to "zod"

* refactor(jwk): infer jwt schema from refinement definition

* fix(handle-resolver): allow resolution errors to propagate
docs(handle-resolver): better handling of DNS resolution errors
fix(handle-resolver): properly handle DOH responses

* fix(did): service endpoint arrays must contain "one or more" element

* refactor(pipe): simplify implementation

* fix(pds): add missing DB indexes

* feat(oauth): Resolve Authorization Server URI through Protected Resource Metadata

* style:(oauth-client): import order

* docs(oauth-provider:redirect-uri): add reference url

* feat(oauth): implement "OAuth Client ID Metadata Document" from draft-parecki-oauth-client-id-metadata-document-latest internet draft

* feat(oauth-client): backport changes from feat-oauth-client

* docs(simple-store): improve comments

* feat(lexicons): add iterable capabilities

* fix(pds): type error in dev mode

* feat(oauth-provider): improved error reporting

* fix(oauth-types): allow insecure issuer during tests

* fix(xrpc-server): allow upload of empty files

* fix: lint

* feat(fetch): keep request reference in errors
feat(fetch): utilities improvements

* fix(pds): allow more than one session token per user

* feat(ozone): improve env validation error messages

* fix(oauth-client): account for DPoP when checking for invalid_token errors

* fixup! feat(fetch): keep request reference in errors feat(fetch): utilities improvements

* fixup! feat(fetch): keep request reference in errors feat(fetch): utilities improvements

* fix(oauth): various validation fixes
feat(oauth): share client_id validation and parsing utilities between client & provider

* feat(dev-env): fix ozone port number

* fix(fetch-node): prevent fetch against invalid domain names

* fix(oauth-provider): add typings for psl dep

* feat(jwk): make type def compatible with TS 4.x

* fix(oauth): fixed various spec compliance
fix(oauth): return "sub" in refresh token response
fix(oauth): limit token validity for third party clients
fix(oauth): hide client image when not trusted

* fix(oauth): lint

* pds: switch changeset to patch, no breaking changes

* changeset and config for new oauth deps

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2024-06-18 15:11:37 -04:00
Matthieu Sieben
97482da17a
Update prettier & eslint (#2373)
* chore(deps): update linting tools

* style(lint): format code
2024-04-03 21:39:11 +02:00
github-actions[bot]
07ec9ea749
Version packages (#2339)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-03-18 18:24:07 -04:00
Matthieu Sieben
f689bd51a2
Build system rework (#2169)
* refactor(crypto): remove circular dependency

* refactor(crypto): expose compress/decompress as part of the DidKeyPlugin interface

* fix(crypto): remove import from private file

* refactor: isolate tsconfig

* fix: remove unused bench file

* chore(repo): remove unused deps

* fix(ozone): properly list dependencies

* fix(services): do lint js files

* fix(services/pds): remove unused deps

* chore(pds): remove bench

* chore(dev-env): remove unused deps

* chore(api): remove bench

* remove unused babel.config.js files

* fix: remove .ts extension from import

* fix(pds): remove imports of src files

* fix(tsconfig): properly list all projects

* fix(dev-env): remove imports of src files

* fix(bsky): remove direct import to crypto src

* fix(api): remove imports to api internals

* chore(build): prevent bundling of built output

* chore(dev): add "dev" script to build in watch mode

* chore(deps): move ts-node dependency where it is actually used

* fix(deps): add dev-env as project dependency

* fix(xrpc-server): properly type kexicon

* fix(bsky): improve typings

* fix(pds): fully type formatRecordEmbedInternal return value

* fix(repo): remove imports from @ipld/car/api

* feat(dev-env): re-export BskyIngester

* fix: properly lint & type jest config & test files

* fix(ci): test after build

* fix(types): use NodeJS.Timeout instead of NodeJS.Timer

* fix(bsky): make types exportable

* fix(ozone): make types exportable

* fix(xrpc-server): make types exportable

* fix(xprc-server): make code compliant with "node" types

* fix(xrpc-server): avoid accessing properties of unknown

* chore(deps): update @types/node

* feat(tsconfig): narrow down available types depending on the package's target environment

* fix(pds): remove unused prop

* fix(bsync): Database's migrator not always initialized

* fix(dev-env): remove unreachable code

* fix(xrpc-server): remove unused import

* fix(xrpc-server): mark header property as abstract

* fix(pds): initialize LeakyTxPlugin's txOver property

* fix(bsky): initialize LeakyTxPlugin's txOver property

* fix(bsky): remove unused migrator from DatabaseCoordinator

* fix(bsky): Properly initialize LabelService's cache property

* fix(ozone): Database's migrator not initialized

* fix(ozone): initialize LeakyTxPlugin's txOver property

* fix(crypto): ignore unused variable error

* feat(tsconfig): use stricter rules

* feat(tsconfig): enable useDefineForClassFields

* feat(xrpc-server): add support for brotli incoming payload

* fix(xrpc-server): properly parse & process content-encoding

* fix(common:stream): always call cb in _transform

* tidy/fix tests and service entrypoints

* Revert "fix(xrpc-server): properly parse & process content-encoding"

This reverts commit 2b1c66e153820d3e128fc839fcc1834d52a66686.

* Revert "feat(xrpc-server): add support for brotli incoming payload"

This reverts commit e710c21e6118214ddf215b0515e68cb87299a952.

* remove special node env for tests (defaults to jest val of "test")

* kill mute sync handler on disconnect

* work around connect-es bug w/ request aborts

* style(crypto): rename imports from uint8arrays

* fix update package-lock

* fix lint

* force hbs files to be bundled as cjs

* fix: use concurrently instead of npm-run-all

npm-run-all seems not to be maintained anymore. Additionally, concurrently better forwards signals to child processes.

* remove concurrently alltogether

* ignore sqlite files in services/pds

* fix verify

* fix verify

* tidy, fix verify

* fix blob diversion test

* build rework changeset

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2024-03-18 17:10:58 -04:00
github-actions[bot]
b286cd1a84
Version packages (#2305)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-03-13 12:50:35 -05:00
github-actions[bot]
1b0b4f93ac
Version packages (#2219)
* Version packages

* Give Mary credit

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eric Bailey <git@esb.lol>
2024-02-29 11:14:25 -06:00
github-actions[bot]
6e6b94be10
Version packages (#2184)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-02-15 20:22:33 -06:00