315 Commits

Author SHA1 Message Date
dan
b6eeb81c6d
Resolve facets in parallel (#2957)
* Detect facets in parallel

* Add changeset
2024-11-08 00:27:03 +00:00
Foysal Ahamed
d6f33b4742
Add events for account and record update/delete/deactivation (#2661)
*  Add events for account and record update/delete/deactivation

*  Add handle change event

*  Reduce account events to 2 types and record events to 1

*  Store metadata from account, identity and record events

*  Add created event for record

*  Add ndd the new events to allowed types in emitEvent

*  Use string value for record op and add tombstone flag to identity event

*  Add active flag on account events

*  Change accountStatus -> status to match with firehose event

*  Make active flag required

* 🚨 fix prettier style issue

*  Track record/account delete and update data in subject status (#2804)

*  Store deleted/updated event data in subject_status

* 🐛 Fix query for recordDeletedAt and recordUpdatedAt

*  Add tombstoned status

*  Move from record to hosting term

*  Add tests for hosting params

*  Update lexicons for hostingStatuses

*  Update snapshots

*  Update snapshots

*  Update snapshots

*  Adjust hosting statuses

* 📝 Add changeset
2024-11-07 22:51:44 +01:00
Foysal Ahamed
c4b5e53957
Ozone instance-wide and user-specific settings (#2905)
*  Settings endpoints are working

* 🧹 Rename file

*  Replace ad-hoc manage roles to match team member roles

* ♻️ Refactor role names

*  Polish up

*  Move to using id for pagination

* 📝 Add changeset

*  Update snapshots

*  Change column order in setting table index and add did in all queries
2024-11-07 22:43:30 +01:00
Matthieu Sieben
839202a3d2
Allow instantiating an api Agent with a string or URL (#2917)
* Allow instantiating an api Agent with a string or URL

* changeset

* Update .changeset/stupid-spiders-buy.md

Co-authored-by: Bigint <69431456+bigint@users.noreply.github.com>

---------

Co-authored-by: Bigint <69431456+bigint@users.noreply.github.com>
2024-11-07 12:22:33 -05:00
rafael
1982693e3e
Use node:crypto instead of noble/curves (#2936)
* Extract verifySignatureWithKey out of verifyJwt

* Accept optional verifySignatureWithKey as param

* Impl. verifySignatureWithKey with native crypto

* Test key validation

* changesets

* build

* build (fix)

* Move verifySig out

* Trigger Build

* Move test

* Remove redundant check

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2024-11-07 13:29:42 -03:00
dan
9e18ab6a35
Fix optimistic getPostThread for URIs with handle (#2946)
* Harden test coverge (add failing test)

* Resolve handle for post thread request

* Changeset
2024-11-04 18:28:39 -06:00
Matthieu Sieben
709ba30157
Properly negotiate response content-encoding (#2852)
* Properly negotiate response content-encoding

* negotiate acceptable encoding and type before building responses

* remove un-necessary async

* typo

* Remove response content-encoding logic

* Avoid using chunked encoding when writing a buffer to the response
2024-11-04 09:40:33 +01:00
Samuel Newman
e680d55ca2
Filter out invalid facets in RichText (#2933)
* add failing test

* simplify test

* check facet was removed

* filter out invalid facets

* changeset
2024-10-31 18:09:49 +00:00
Matthieu Sieben
9ffeb5216a
Allow retrying proxied requests (#2850)
* Allow retrying proxied requests

* fix tests

* remove un-necessary code

* Only retry on socket errors
2024-10-31 00:21:05 -04:00
github-actions[bot]
7ea0fc146b
Version packages (#2930)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-30 18:36:37 -05:00
Daniel Holmgren
73f40e63ab
Add threat signatures to ozone repo views (#2931)
* add threat signatures to ozone repo views

* explicitly add to views
2024-10-30 18:30:41 -05:00
devin ivy
209238769c
Add app.bsky.unspecced.getConfig endpoint (#2918)
* lexicon: add app.bsky.unspecced.getConfig for client runtime config

* bsky: add app.bsky.unspecced.getConfig w/ new env for checkEmailConfirmed

* changeset
2024-10-30 19:09:54 -04:00
github-actions[bot]
d9fcaf8a78
Version packages (#2928)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-30 18:07:13 -05:00
Foysal Ahamed
19e36afb2c
Add collection and subjectType filters to queryEvents and queryStatuses (#2914)
*  Add collection and subjectType filters to queryEvents and queryStatuses

* 📝 Add changeset

* ♻️ Refactor or query builder
2024-10-30 16:20:38 +01:00
github-actions[bot]
91bd63ba38
Version packages (#2925)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-29 18:20:10 -05:00
Daniel Holmgren
c1b0e176ad
updateHandle changeset (#2924)
changset
2024-10-29 17:58:56 -05:00
github-actions[bot]
c307a75db1
Version packages (#2889)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-24 11:32:50 -05:00
Daniel Holmgren
d605577c25
@atproto/sync: avoid parsing commits with no relevant ops (#2906)
* avoid parsing commits with no relevant ops

* changeset
2024-10-24 11:07:02 -05:00
Matthieu Sieben
8f2b80a0dc
Better report invalid content-encoding errors (#2902) 2024-10-21 14:22:29 +02:00
Matthieu Sieben
9d40ccbb69
Various OAuth related fixes (#2871)
* wip

* tidy

* tidy

* tidy

* Update packages/oauth/oauth-client/src/session-getter.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* fix combineSignals

* tidy

* tidy

* improve typing of atprotoScopeSchema

* stronger typings

* tidy

* ci

* Fix cors error

* downgrade ioredis dependency

* fix ioredis version

* tidy

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2024-10-18 20:23:33 +02:00
Matthieu Sieben
7f26b17652
Add OAuth tests (#2874)
* Improve error message when using invalid client_id during code exchange

* Extract SPA example OAuth client in own package

* wip

* remove dependency on get-port

* Properly configure jest to only transpile "get-port" from node_modules

https://jestjs.io/docs/configuration#transformignorepatterns-arraystring

* Use dynamically assigned port number during tests

* use puppeteer to run tests

* remove login input "id" attribute

* code style

* add missing declaration

* tidy

* headless

* remove get-port dependency

* fix tests/proxied/admin.test.ts

* fix tests

* Allow unsecure oauth providers through configuration

* transpile "lande" during ozone tests

* Cache Puppeteer browser binaries

* Use puppeteer cache during all workflow steps

* remove use of set-output

* use get-port in xrpc-server tests

* Renamed to allowHttp

* tidy

* tidy
2024-10-18 15:40:05 +02:00
github-actions[bot]
4f6f57a33e
Version packages (#2870)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-11 10:44:20 +02:00
Foysal Ahamed
22d039a229
Ozone sets (#2636)
*  Initial implementation of sets api on ozone

*  Introduce sortDirection to querySets

* 🧹 Cleanup and refactor

*  Align setView for response

* ♻️ Rename and add specific error

* 🐛 Cleanup unnecessary check that is covered by lexicon

*  Rename remove to delete and add set suffix

*  Use id and createdAt for values pagination

*  Add index on createdAt for query perf and other cleanups

* 🐛 Set createdAt when inserting values

* 📝 Add changeset

*  Add index on setId and createdAt
2024-10-08 19:16:09 +02:00
github-actions[bot]
3e1ae8d1d5
Version packages (#2861)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-04 15:07:43 -04:00
Matthieu Sieben
80450cbf2c
Do not use HTTP2 connection when performing "safe fetch" HTTP requests (#2865) 2024-10-04 18:55:15 +02:00
Matthieu Sieben
08ed0a5a91
Fix build (#2862)
add missing ts dependency
2024-10-04 11:40:50 +02:00
Matthieu Sieben
72549f4422
Allow using a handle as "actor" param in app.bsky.graph.getLists (#2853)
* Allow using a handle as "actor" param in app.bsky.graph.getLists

* tests
2024-10-04 07:18:38 +02:00
github-actions[bot]
556c5a549c
Version packages (#2859)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-03 20:36:45 -05:00
Eric Bailey
a0531ce429
Add check for external media within RecordWithMedia (#2857) 2024-10-03 20:24:00 -05:00
github-actions[bot]
600fea65d4
Version packages (#2856)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-03 14:07:55 -05:00
Matthieu Sieben
8943c10082
Disable use of HTTP2 when checking SSRF IP (#2854) 2024-10-03 14:03:48 -05:00
Daniel Holmgren
df14df522b
Ozone signature methods (#2855)
* lexicons

* fix typo & add proxy routes

* changeset
2024-10-03 14:00:12 -05:00
github-actions[bot]
a611a5fe56
Version packages (#2846)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-01 20:13:26 +02:00
Foysal Ahamed
a2bad977a8
Ozone batch repo and record getters (#2836)
*  Add getRepos and getRecords endpoints for bulk fetching

*  Fix issues and add tests for get repos and get records

*  Use the right lxm

* 🐛 Revert changes in lockfile

*  Add getAccountInfos in PDS

* 🐛 Fix type def for repo and record view detail

*  Update snapshots

*  Update snapshots

*  Consolidate error type for com.atproto and tools.ozone getRecord error type

* 🧹 Cleanup

*  Update snapshots

*  Update snapshots

*  Changeset
2024-10-01 19:37:23 +02:00
Matthieu Sieben
1226ed2682
Do not display the client_name of untrusted clients (#2847)
* Do not display the client_name of untrusted clients

* do not show client id of trusted clients
2024-10-01 19:08:26 +02:00
Matthieu Sieben
4098d98901
Default to unencoded responses (#2834)
* Allow defaulting to unencoded responses when proxying client requests that do not specify accept-encoding
* fix content-encoding negotiation
2024-10-01 10:43:15 +02:00
github-actions[bot]
6593fdc3f4
Version packages (#2812)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-27 13:04:34 -05:00
Matthieu Sieben
eb20ff64a2
ponyfill URL.canParse (#2835) 2024-09-27 17:03:29 +02:00
Samuel Newman
2676206e42
Pinned posts (#2771)
* pinned posts lexicon

* codegen

* change lexicon, different approach

* codegen 2

* dataplane db migration

* move pinned post lexicon to right place

* add pinned posts optionally to getAuthorFeed

* remove type modification

* Clarify naming, add viewer state, add tests

* return pinnedPost with profileViewDetailed

* allow pinned replies in `posts_and_author_threads`

* clearer variable naming

* annotate type of `items`

* boolean --> varchar

* reuse authorDid in viewerPinned

* simplify test

* make pinned post not top post in test

* update snapshot

* changeset

---------

Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: dholms <dtholmgren@gmail.com>
2024-09-26 18:26:45 -05:00
Matthieu Sieben
ed325d863c
OAuth spec alignment (#2755)
* Improve reporting of metadata validation error
* Properly validate client metadata scope
* Allow loopback clients to define their scopes through client_id query parameters
* Require definition of "scope" in client metadata document
* Restrict the value used as code_challenge_methods_supported
* Remove `plain` from `code_challenge_methods_supported`
* Prevent use of empty string in unsupported oidc request parameters
* Centralize parsing of client metadata error
* Enfore code_challenge_method=S256 request parameter
* Improve error description in case of invalid loopback client_id
* Enfore single scope query param in loopback clients
* Disable request params scopes defaulting to client metadata scope
* Centralize loopback client validation logic
* add assertion utils for client ids
* Improve invalid client_id error messages from BrowserOAuthClient.from()
* Use scope from client metadata as default value
* Improve client side validation of client metadata
* Allow fetching of source maps files from browser debugger
* Use the clientId to configure the OAuth client
* Allow native clients to use https: redirect uris
* Explicitely forbid MTLS client auth method
* Improve error feedback in case of invalid client_id domain name
* Remove un-spec'ed restrictions on redirect_uris based on the client_uri
* Do not strip query string from URL after oauth redirect in fragment mode
* Add missing "expires_in" property to OAuthParResponse type definition
* Allow non canonical urls to be used as client ID
* Allow client metadata to contain other return type values than "code"
* Properly validate request_uri request parameter
* Improve parsing and validation of client_id's
* Return "invalid_client" on invalid client credentials
* improved error management & reporting
* performance improvement
* Allow loopback client ids to omit the (empty) path parameter

Co-authored-by: devin ivy <devinivy@gmail.com>
2024-09-26 14:07:08 +02:00
dan
87a1f24262
Add fast path skipping grapheme counting (#2817)
* Cache length calculations between min and max

* Harden grapheme counter tests

* Add fast paths

* Code style tweaks

* changeset

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2024-09-25 20:17:42 +09:00
Matthieu Sieben
b298bfd280
Prevent PDS crash when catchall proxy request are cancelled (#2824)
* Prevent PDS crash when catchall proxy request are cancelled

* fixes

* ensure stream destruction
2024-09-24 11:55:29 -05:00
Matthieu Sieben
a07b21151f
PDS pipethrough optimizations (#2770)
* Micro optimization in request proxying

* Request NSID parsing optimization

* DID document parsing optimization

* remove un-necessary call to next()

* Allow HandlerPipeThrough to be used with streams

* Refactor pipethrough to work with streams

* Expose "unicastLookup" DNS lookup and "isUnicastIp" utilities

* Use a hardened, HTTP2 compatible, client to perform proxied requests

* changeset

* tidy

* Properly handle compressed streams

* tidy

* update @types/node

* refactor

* Improved error management

* Expose parseContentEncoding() util

* use pipeline from nodejs

* Avoid decoding in read-after-write (if possible)

* Various fixes

* Return Buffer instance from streamToBytes

* fixes

* Add omit() utility

* tidy

* lint

* typo

* Use Buffer instead of ArrayBuffer form pipe through handler result

* optimization

* tidy

* refactor

* increase highWaterMark

* remove un-necessary type check

* Use undici.request where more relevant

* Improve soc in fetch utils

* feedback

* fidy

* tidy

* test refactor

* safer fetch

* changeset

* expose and re-use extractUrl util

* small optimizations

* tidy

* optimization

* build branch

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2024-09-19 18:24:20 -05:00
Eric Bailey
a06634ae57
Email templates at (#2813)
* Add @ to handles, update PLC copy

* Changeset

* lint

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2024-09-12 11:57:41 -05:00
Eric Bailey
922b94ce37
Update email templates (#2767)
* Update email templates

* Update PLC

* Update test with new email string

* Format

* One more test update

* Use handle instead of identifier to match entryway

* Changeset
2024-09-12 10:53:31 -05:00
github-actions[bot]
85c85350d1
Version packages (#2791)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-11 18:30:47 -05:00
Eric Bailey
33aa0c7222
NUX API (#2810)
* Codegen

* Explicitly add Zod (already a peer dep) and validation to api

* Add Nux methods

* Match naming convention

* Remove id, it won't be used

* Add tests

* Use id instead of name, little clearer

* Update API contracts

* Update tests

* Changeset

* Don't mutate
2024-09-11 18:25:05 -05:00
Foysal Ahamed
e6bd5aecce
📝 Add changeset for acknowledgeAccountSubjects flag (#2807) 2024-09-12 00:03:52 +02:00
Matthieu Sieben
98711a147a
fix(xrpc-server): properly parse & process content-encoding (#2464)
* fix(xrpc-server): properly parse & process content-encoding

* Minor optimization

* code style
2024-09-11 09:46:18 +02:00
Matthieu Sieben
cb4abbb673
Properly validate atproto did:web (#2776)
* Properly validate atproto did:web

* explicit why there is no protection against localhost fetches in did:web resolver
2024-09-05 13:01:48 +02:00