457 Commits

Author SHA1 Message Date
Matthieu Sieben
9ffeb5216a
Allow retrying proxied requests (#2850)
* Allow retrying proxied requests

* fix tests

* remove un-necessary code

* Only retry on socket errors
2024-10-31 00:21:05 -04:00
github-actions[bot]
7ea0fc146b
Version packages (#2930)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-30 18:36:37 -05:00
Daniel Holmgren
73f40e63ab
Add threat signatures to ozone repo views (#2931)
* add threat signatures to ozone repo views

* explicitly add to views
2024-10-30 18:30:41 -05:00
devin ivy
209238769c
Add app.bsky.unspecced.getConfig endpoint (#2918)
* lexicon: add app.bsky.unspecced.getConfig for client runtime config

* bsky: add app.bsky.unspecced.getConfig w/ new env for checkEmailConfirmed

* changeset
2024-10-30 19:09:54 -04:00
github-actions[bot]
d9fcaf8a78
Version packages (#2928)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-30 18:07:13 -05:00
Foysal Ahamed
19e36afb2c
Add collection and subjectType filters to queryEvents and queryStatuses (#2914)
*  Add collection and subjectType filters to queryEvents and queryStatuses

* 📝 Add changeset

* ♻️ Refactor or query builder
2024-10-30 16:20:38 +01:00
github-actions[bot]
91bd63ba38
Version packages (#2925)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-29 18:20:10 -05:00
Daniel Holmgren
c1b0e176ad
updateHandle changeset (#2924)
changset
2024-10-29 17:58:56 -05:00
github-actions[bot]
c307a75db1
Version packages (#2889)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-24 11:32:50 -05:00
Daniel Holmgren
d605577c25
@atproto/sync: avoid parsing commits with no relevant ops (#2906)
* avoid parsing commits with no relevant ops

* changeset
2024-10-24 11:07:02 -05:00
Matthieu Sieben
8f2b80a0dc
Better report invalid content-encoding errors (#2902) 2024-10-21 14:22:29 +02:00
Matthieu Sieben
9d40ccbb69
Various OAuth related fixes (#2871)
* wip

* tidy

* tidy

* tidy

* Update packages/oauth/oauth-client/src/session-getter.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* fix combineSignals

* tidy

* tidy

* improve typing of atprotoScopeSchema

* stronger typings

* tidy

* ci

* Fix cors error

* downgrade ioredis dependency

* fix ioredis version

* tidy

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2024-10-18 20:23:33 +02:00
Matthieu Sieben
7f26b17652
Add OAuth tests (#2874)
* Improve error message when using invalid client_id during code exchange

* Extract SPA example OAuth client in own package

* wip

* remove dependency on get-port

* Properly configure jest to only transpile "get-port" from node_modules

https://jestjs.io/docs/configuration#transformignorepatterns-arraystring

* Use dynamically assigned port number during tests

* use puppeteer to run tests

* remove login input "id" attribute

* code style

* add missing declaration

* tidy

* headless

* remove get-port dependency

* fix tests/proxied/admin.test.ts

* fix tests

* Allow unsecure oauth providers through configuration

* transpile "lande" during ozone tests

* Cache Puppeteer browser binaries

* Use puppeteer cache during all workflow steps

* remove use of set-output

* use get-port in xrpc-server tests

* Renamed to allowHttp

* tidy

* tidy
2024-10-18 15:40:05 +02:00
github-actions[bot]
4f6f57a33e
Version packages (#2870)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-11 10:44:20 +02:00
Foysal Ahamed
22d039a229
Ozone sets (#2636)
*  Initial implementation of sets api on ozone

*  Introduce sortDirection to querySets

* 🧹 Cleanup and refactor

*  Align setView for response

* ♻️ Rename and add specific error

* 🐛 Cleanup unnecessary check that is covered by lexicon

*  Rename remove to delete and add set suffix

*  Use id and createdAt for values pagination

*  Add index on createdAt for query perf and other cleanups

* 🐛 Set createdAt when inserting values

* 📝 Add changeset

*  Add index on setId and createdAt
2024-10-08 19:16:09 +02:00
github-actions[bot]
3e1ae8d1d5
Version packages (#2861)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-04 15:07:43 -04:00
Matthieu Sieben
80450cbf2c
Do not use HTTP2 connection when performing "safe fetch" HTTP requests (#2865) 2024-10-04 18:55:15 +02:00
Matthieu Sieben
08ed0a5a91
Fix build (#2862)
add missing ts dependency
2024-10-04 11:40:50 +02:00
Matthieu Sieben
72549f4422
Allow using a handle as "actor" param in app.bsky.graph.getLists (#2853)
* Allow using a handle as "actor" param in app.bsky.graph.getLists

* tests
2024-10-04 07:18:38 +02:00
github-actions[bot]
556c5a549c
Version packages (#2859)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-03 20:36:45 -05:00
Eric Bailey
a0531ce429
Add check for external media within RecordWithMedia (#2857) 2024-10-03 20:24:00 -05:00
github-actions[bot]
600fea65d4
Version packages (#2856)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-03 14:07:55 -05:00
Matthieu Sieben
8943c10082
Disable use of HTTP2 when checking SSRF IP (#2854) 2024-10-03 14:03:48 -05:00
Daniel Holmgren
df14df522b
Ozone signature methods (#2855)
* lexicons

* fix typo & add proxy routes

* changeset
2024-10-03 14:00:12 -05:00
github-actions[bot]
a611a5fe56
Version packages (#2846)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-01 20:13:26 +02:00
Foysal Ahamed
a2bad977a8
Ozone batch repo and record getters (#2836)
*  Add getRepos and getRecords endpoints for bulk fetching

*  Fix issues and add tests for get repos and get records

*  Use the right lxm

* 🐛 Revert changes in lockfile

*  Add getAccountInfos in PDS

* 🐛 Fix type def for repo and record view detail

*  Update snapshots

*  Update snapshots

*  Consolidate error type for com.atproto and tools.ozone getRecord error type

* 🧹 Cleanup

*  Update snapshots

*  Update snapshots

*  Changeset
2024-10-01 19:37:23 +02:00
Matthieu Sieben
1226ed2682
Do not display the client_name of untrusted clients (#2847)
* Do not display the client_name of untrusted clients

* do not show client id of trusted clients
2024-10-01 19:08:26 +02:00
Matthieu Sieben
4098d98901
Default to unencoded responses (#2834)
* Allow defaulting to unencoded responses when proxying client requests that do not specify accept-encoding
* fix content-encoding negotiation
2024-10-01 10:43:15 +02:00
github-actions[bot]
6593fdc3f4
Version packages (#2812)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-27 13:04:34 -05:00
Matthieu Sieben
eb20ff64a2
ponyfill URL.canParse (#2835) 2024-09-27 17:03:29 +02:00
Samuel Newman
2676206e42
Pinned posts (#2771)
* pinned posts lexicon

* codegen

* change lexicon, different approach

* codegen 2

* dataplane db migration

* move pinned post lexicon to right place

* add pinned posts optionally to getAuthorFeed

* remove type modification

* Clarify naming, add viewer state, add tests

* return pinnedPost with profileViewDetailed

* allow pinned replies in `posts_and_author_threads`

* clearer variable naming

* annotate type of `items`

* boolean --> varchar

* reuse authorDid in viewerPinned

* simplify test

* make pinned post not top post in test

* update snapshot

* changeset

---------

Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: dholms <dtholmgren@gmail.com>
2024-09-26 18:26:45 -05:00
Matthieu Sieben
ed325d863c
OAuth spec alignment (#2755)
* Improve reporting of metadata validation error
* Properly validate client metadata scope
* Allow loopback clients to define their scopes through client_id query parameters
* Require definition of "scope" in client metadata document
* Restrict the value used as code_challenge_methods_supported
* Remove `plain` from `code_challenge_methods_supported`
* Prevent use of empty string in unsupported oidc request parameters
* Centralize parsing of client metadata error
* Enfore code_challenge_method=S256 request parameter
* Improve error description in case of invalid loopback client_id
* Enfore single scope query param in loopback clients
* Disable request params scopes defaulting to client metadata scope
* Centralize loopback client validation logic
* add assertion utils for client ids
* Improve invalid client_id error messages from BrowserOAuthClient.from()
* Use scope from client metadata as default value
* Improve client side validation of client metadata
* Allow fetching of source maps files from browser debugger
* Use the clientId to configure the OAuth client
* Allow native clients to use https: redirect uris
* Explicitely forbid MTLS client auth method
* Improve error feedback in case of invalid client_id domain name
* Remove un-spec'ed restrictions on redirect_uris based on the client_uri
* Do not strip query string from URL after oauth redirect in fragment mode
* Add missing "expires_in" property to OAuthParResponse type definition
* Allow non canonical urls to be used as client ID
* Allow client metadata to contain other return type values than "code"
* Properly validate request_uri request parameter
* Improve parsing and validation of client_id's
* Return "invalid_client" on invalid client credentials
* improved error management & reporting
* performance improvement
* Allow loopback client ids to omit the (empty) path parameter

Co-authored-by: devin ivy <devinivy@gmail.com>
2024-09-26 14:07:08 +02:00
dan
87a1f24262
Add fast path skipping grapheme counting (#2817)
* Cache length calculations between min and max

* Harden grapheme counter tests

* Add fast paths

* Code style tweaks

* changeset

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2024-09-25 20:17:42 +09:00
Matthieu Sieben
b298bfd280
Prevent PDS crash when catchall proxy request are cancelled (#2824)
* Prevent PDS crash when catchall proxy request are cancelled

* fixes

* ensure stream destruction
2024-09-24 11:55:29 -05:00
Matthieu Sieben
a07b21151f
PDS pipethrough optimizations (#2770)
* Micro optimization in request proxying

* Request NSID parsing optimization

* DID document parsing optimization

* remove un-necessary call to next()

* Allow HandlerPipeThrough to be used with streams

* Refactor pipethrough to work with streams

* Expose "unicastLookup" DNS lookup and "isUnicastIp" utilities

* Use a hardened, HTTP2 compatible, client to perform proxied requests

* changeset

* tidy

* Properly handle compressed streams

* tidy

* update @types/node

* refactor

* Improved error management

* Expose parseContentEncoding() util

* use pipeline from nodejs

* Avoid decoding in read-after-write (if possible)

* Various fixes

* Return Buffer instance from streamToBytes

* fixes

* Add omit() utility

* tidy

* lint

* typo

* Use Buffer instead of ArrayBuffer form pipe through handler result

* optimization

* tidy

* refactor

* increase highWaterMark

* remove un-necessary type check

* Use undici.request where more relevant

* Improve soc in fetch utils

* feedback

* fidy

* tidy

* test refactor

* safer fetch

* changeset

* expose and re-use extractUrl util

* small optimizations

* tidy

* optimization

* build branch

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2024-09-19 18:24:20 -05:00
Eric Bailey
a06634ae57
Email templates at (#2813)
* Add @ to handles, update PLC copy

* Changeset

* lint

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2024-09-12 11:57:41 -05:00
Eric Bailey
922b94ce37
Update email templates (#2767)
* Update email templates

* Update PLC

* Update test with new email string

* Format

* One more test update

* Use handle instead of identifier to match entryway

* Changeset
2024-09-12 10:53:31 -05:00
github-actions[bot]
85c85350d1
Version packages (#2791)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-11 18:30:47 -05:00
Eric Bailey
33aa0c7222
NUX API (#2810)
* Codegen

* Explicitly add Zod (already a peer dep) and validation to api

* Add Nux methods

* Match naming convention

* Remove id, it won't be used

* Add tests

* Use id instead of name, little clearer

* Update API contracts

* Update tests

* Changeset

* Don't mutate
2024-09-11 18:25:05 -05:00
Foysal Ahamed
e6bd5aecce
📝 Add changeset for acknowledgeAccountSubjects flag (#2807) 2024-09-12 00:03:52 +02:00
Matthieu Sieben
98711a147a
fix(xrpc-server): properly parse & process content-encoding (#2464)
* fix(xrpc-server): properly parse & process content-encoding

* Minor optimization

* code style
2024-09-11 09:46:18 +02:00
Matthieu Sieben
cb4abbb673
Properly validate atproto did:web (#2776)
* Properly validate atproto did:web

* explicit why there is no protection against localhost fetches in did:web resolver
2024-09-05 13:01:48 +02:00
github-actions[bot]
0a37a3cc56
Version packages (#2788)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-04 20:22:22 -05:00
Daniel Holmgren
b15dec2f4f
Atproto sync package (#2752)
* first pass/port

* reworking

* authenticated commit parsing

* authenticate identity evts

* some testing

* tidy & add firehose to queue

* error handling

* fix test

* refactor sync queue + some tests

* fix race in sync queue

* rm firehose from syncqueue

* add tests for queue utils

* README

* lint readme

* filter before parsing

* pr feedback

* small fix

* changesets

* fix type

* Rework dataplane subscription (#2766)

* working sync package into appview subscription

* add restart method to subscription for tests

* fix another test

* tidy subscription utils/files

* remove dupe property

* tidy after merge

* fix start cursor on subscription

* tweak process full subscription logic

* fixes
2024-09-04 20:18:16 -05:00
Foysal Ahamed
642c7ae968
Improve query perf on moderation event (#2787)
*  Add index on subjectDid and use subjectDid to speed up query

* 📝 Add changeset
2024-09-04 19:48:16 -04:00
github-actions[bot]
71305e8595
Version packages (#2764)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-04 14:23:00 +02:00
Foysal Ahamed
e4d41d66fa
❇️ Template language (#2780)
*  Throw specific error for duplicate template name

* 🧹 Cleanup console

*  Throw duplicate template name error from update too

*  Add language to templates

* 📝 Add changeset

*  Add missing event type

*  Add language format in lexicon and error checker in util

* 🚨 fix linter issues
2024-09-04 12:42:39 +02:00
Foysal Ahamed
325859b8bf
📝 Add changeset for moderation subject's embed type tagging (#2762)
📝 Add changeset
2024-08-29 14:14:15 -04:00
github-actions[bot]
6bc7faf087
Version packages (#2761)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-28 19:13:54 -04:00
devin ivy
80ada8f476
Video lexicons and appview views (#2751)
* lexicon: initial lexicons for video embeds in bsky app

* lexicon: fix video caption file size limit

* codegen

* appview: stub out video embed view logic

* api prerelease

* api prerelease

* lexicon: video upload/processing lexicons

* tidy

* lexicon: app.bsky.video lexicons for uploads

* codegen

* api prerelease

* appview: present video embeds on posts

* appview: snaps

* changeset

* appview: fix wiring of video url config
2024-08-28 19:03:35 -04:00