365 Commits

Author SHA1 Message Date
Matthieu Sieben
9d40ccbb69
Various OAuth related fixes (#2871)
* wip

* tidy

* tidy

* tidy

* Update packages/oauth/oauth-client/src/session-getter.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* fix combineSignals

* tidy

* tidy

* improve typing of atprotoScopeSchema

* stronger typings

* tidy

* ci

* Fix cors error

* downgrade ioredis dependency

* fix ioredis version

* tidy

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2024-10-18 20:23:33 +02:00
Matthieu Sieben
7f26b17652
Add OAuth tests (#2874)
* Improve error message when using invalid client_id during code exchange

* Extract SPA example OAuth client in own package

* wip

* remove dependency on get-port

* Properly configure jest to only transpile "get-port" from node_modules

https://jestjs.io/docs/configuration#transformignorepatterns-arraystring

* Use dynamically assigned port number during tests

* use puppeteer to run tests

* remove login input "id" attribute

* code style

* add missing declaration

* tidy

* headless

* remove get-port dependency

* fix tests/proxied/admin.test.ts

* fix tests

* Allow unsecure oauth providers through configuration

* transpile "lande" during ozone tests

* Cache Puppeteer browser binaries

* Use puppeteer cache during all workflow steps

* remove use of set-output

* use get-port in xrpc-server tests

* Renamed to allowHttp

* tidy

* tidy
2024-10-18 15:40:05 +02:00
Matthieu Sieben
fabc8a9381
Update typescript to version 5.6.2 (#2863) 2024-10-11 14:05:53 +02:00
github-actions[bot]
4f6f57a33e
Version packages (#2870)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-11 10:44:20 +02:00
Foysal Ahamed
22d039a229
Ozone sets (#2636)
*  Initial implementation of sets api on ozone

*  Introduce sortDirection to querySets

* 🧹 Cleanup and refactor

*  Align setView for response

* ♻️ Rename and add specific error

* 🐛 Cleanup unnecessary check that is covered by lexicon

*  Rename remove to delete and add set suffix

*  Use id and createdAt for values pagination

*  Add index on createdAt for query perf and other cleanups

* 🐛 Set createdAt when inserting values

* 📝 Add changeset

*  Add index on setId and createdAt
2024-10-08 19:16:09 +02:00
github-actions[bot]
3e1ae8d1d5
Version packages (#2861)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-04 15:07:43 -04:00
Matthieu Sieben
08ed0a5a91
Fix build (#2862)
add missing ts dependency
2024-10-04 11:40:50 +02:00
github-actions[bot]
556c5a549c
Version packages (#2859)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-03 20:36:45 -05:00
github-actions[bot]
600fea65d4
Version packages (#2856)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-03 14:07:55 -05:00
Daniel Holmgren
df14df522b
Ozone signature methods (#2855)
* lexicons

* fix typo & add proxy routes

* changeset
2024-10-03 14:00:12 -05:00
github-actions[bot]
a611a5fe56
Version packages (#2846)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-01 20:13:26 +02:00
Foysal Ahamed
a2bad977a8
Ozone batch repo and record getters (#2836)
*  Add getRepos and getRecords endpoints for bulk fetching

*  Fix issues and add tests for get repos and get records

*  Use the right lxm

* 🐛 Revert changes in lockfile

*  Add getAccountInfos in PDS

* 🐛 Fix type def for repo and record view detail

*  Update snapshots

*  Update snapshots

*  Consolidate error type for com.atproto and tools.ozone getRecord error type

* 🧹 Cleanup

*  Update snapshots

*  Update snapshots

*  Changeset
2024-10-01 19:37:23 +02:00
Foysal Ahamed
cafa8a15c0
Add index on reviewState column on moderation_subject_status table (#2848) 2024-10-01 19:07:55 +02:00
github-actions[bot]
6593fdc3f4
Version packages (#2812)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-27 13:04:34 -05:00
Samuel Newman
2676206e42
Pinned posts (#2771)
* pinned posts lexicon

* codegen

* change lexicon, different approach

* codegen 2

* dataplane db migration

* move pinned post lexicon to right place

* add pinned posts optionally to getAuthorFeed

* remove type modification

* Clarify naming, add viewer state, add tests

* return pinnedPost with profileViewDetailed

* allow pinned replies in `posts_and_author_threads`

* clearer variable naming

* annotate type of `items`

* boolean --> varchar

* reuse authorDid in viewerPinned

* simplify test

* make pinned post not top post in test

* update snapshot

* changeset

---------

Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: dholms <dtholmgren@gmail.com>
2024-09-26 18:26:45 -05:00
Eric Bailey
319aa7cf6d
Add isFallback to getSuggestedFollowsByActor method (#2805)
* Add isFallback to `getSuggestedFollowsByActor`

Inferred based on returned `relativeToDid` from the suggestions
response.

* Integrate new params

* Fix logic
2024-09-13 12:37:15 -05:00
github-actions[bot]
85c85350d1
Version packages (#2791)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-11 18:30:47 -05:00
Eric Bailey
33aa0c7222
NUX API (#2810)
* Codegen

* Explicitly add Zod (already a peer dep) and validation to api

* Add Nux methods

* Match naming convention

* Remove id, it won't be used

* Add tests

* Use id instead of name, little clearer

* Update API contracts

* Update tests

* Changeset

* Don't mutate
2024-09-11 18:25:05 -05:00
Foysal Ahamed
62a8225f36
Optionally ack all open subjects from the author with takedown event and get all mod subjects of a user (#2793)
*  Add acknowledgeAllSubjectsOfAccount flag with takedown event

* 📝 better documentation of forAccount and subject params

* 🧹 Cleanup tests

*  Change wording according to review

*  Refactor tests

* ♻️ Rename acknowledge flag
2024-09-11 18:26:53 +02:00
devin ivy
71785d31f7
Ozone: pass through unknown account details (#2794)
* ozone: pass through account info details that may not be known yet

* tidy, build
2024-09-05 21:22:48 -04:00
github-actions[bot]
0a37a3cc56
Version packages (#2788)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-04 20:22:22 -05:00
Foysal Ahamed
642c7ae968
Improve query perf on moderation event (#2787)
*  Add index on subjectDid and use subjectDid to speed up query

* 📝 Add changeset
2024-09-04 19:48:16 -04:00
github-actions[bot]
71305e8595
Version packages (#2764)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-09-04 14:23:00 +02:00
Foysal Ahamed
e4d41d66fa
❇️ Template language (#2780)
*  Throw specific error for duplicate template name

* 🧹 Cleanup console

*  Throw duplicate template name error from update too

*  Add language to templates

* 📝 Add changeset

*  Add missing event type

*  Add language format in lexicon and error checker in util

* 🚨 fix linter issues
2024-09-04 12:42:39 +02:00
Foysal Ahamed
8252c652e8
🐛 Fix appealed param's usage (#2777)
🐛 Depend on false value instead of null since the input is sent through GET request for queryStatuses
2024-09-02 19:10:54 -04:00
Daniel Holmgren
bfbac24312
Repo write commit metadata (#2745)
* schemas & codegen

* impl

* bundle commit & rev
2024-08-30 13:18:43 -05:00
devin ivy
befebc0e98
Revise repo write validation for unknown Lexicons (#2748)
* lexicon: validation status result from repo writes

* pds: return  validation status from repo writes, write results from applyWrites

* tidy
2024-08-30 12:59:55 -05:00
Foysal Ahamed
372ed4c519
Apply embed specific tag on subjects for video, image and external (#2703)
*  Refactor subject tagging to facilitate video content tagging

* ♻️ Refactor tag check

*  Fix tagging logic

* ♻️ Refactor content tagger and fix image content type check

*  Add embed tag check for video and external

*  Add tagging for both media and image embed
2024-08-29 18:23:48 +02:00
github-actions[bot]
6bc7faf087
Version packages (#2761)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-28 19:13:54 -04:00
devin ivy
80ada8f476
Video lexicons and appview views (#2751)
* lexicon: initial lexicons for video embeds in bsky app

* lexicon: fix video caption file size limit

* codegen

* appview: stub out video embed view logic

* api prerelease

* api prerelease

* lexicon: video upload/processing lexicons

* tidy

* lexicon: app.bsky.video lexicons for uploads

* codegen

* api prerelease

* appview: present video embeds on posts

* appview: snaps

* changeset

* appview: fix wiring of video url config
2024-08-28 19:03:35 -04:00
github-actions[bot]
a1d8c77edd
Version packages (#2738)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-27 14:03:39 -04:00
Eric Bailey
a8e1f9000d
Return ThreadgateView on response from getPostThread (#2737)
* Return `ThreadgateView` on response from `getPostThread`

* Changeset

* Format

* Add to test

* Clean up logic

* Use suggestion from Dan
2024-08-22 17:00:01 -05:00
github-actions[bot]
f70bd6a9dc
Version packages (#2736)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-22 09:54:41 -07:00
Hailey
4ab2483547
Add quoteCount to embed view (#2735)
Co-authored-by: devin ivy <devinivy@gmail.com>
2024-08-22 09:49:06 -07:00
Matthieu Sieben
5fef534610 codegen 2024-08-22 11:55:36 +02:00
github-actions[bot]
1572058887
Version packages (#2732)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-21 14:42:01 -05:00
Eric Bailey
aba664fbdf
Detached QPs and hidden replies (#2675)
* Add new postgate lex, hiddeReplies to threadgate, codegen

* Add protobufs

* Add to mock dataplane

* Add matching postgate method to feed hydration methods

* Add to getRecord

* Add to HydrationState

* Fix typo

* Add to mergeStates, fetch embeds in threads

* Integrate into embed views

* Add test for QPs in threads

* Add feed test

* Fix naming convention in protos

* Add #viewRemoved record view, rename postgate.json

* Integrate new view

* Filter hidden replies from feeds

* Filter out replies at the handler level, do not filter for author feeds

* Fix lint

* Move hidden reply check to view layer

* Reduce, reuse, recycle

* Rename to lowercase

* Rename layer vars

* Add quote gate props to postgate (#2693)

* Add quote gate props to postgate

* Consistent naming

* Fix record structure

* Codegen

* Show hidden replies in author feed

* Allow reposts of hidden replies

* Lex and codegen

* Add violates_quote_gate to proto

* Consistent naming, codegen

* Integrate violatesQuotegate and canQuotepost

* Remove rules, codegen

* Hydrate all postgates for all requested posts

* Match other impl

* Add test, need to split these out

* Format

* Hydrate first nested embeds too

* Add postgate test suite

* Add violatesQuoteGate to dataplane

* Ingest and set violatesQuoteGate, return on meta

* Return removed embed for quotes that violate gate

* Add test

* Dedupe URIs before fetching postgates

* Update snaps

* Snap

* Format

* Updating naming conventions for postgate-related attributes

* Correct naming

* Consistency

* Proto too

* Rename to viewDetached

* Codegen

* Rename everything

* Codegen

* Quotes that violate a quote gate can still be quoted themselves

* Couple more renames

* Snaps

* Ensure reply ref is tombstoned for hidden replies

* Split out hidden replies tests and create fresh fixture

* Hydrate threadgates for reply notifications, filter hidden replies

* Remove snap

* Add flaky test

* Rename violatesEmbeddingRules

* Fix flaky test

* Only write to db if violatesEmbeddingRules is true

* DRY up post uri -> gate uri logic

* isThreadgateListRule

* Don't share users object between tests

* No pascal

* Remove default params

* Find -> some

* canQuotepost -> canEmbed, remove optional

* Fix quoteee typo

* await follows

* Throw in post uri -> gate utils

* Ensure fetch threadgates for reply roots

* Don't hydrate threadgates twice

* DRY up uri -> did parsing

* Clean up parsePostgate logic

* Format

* Revert change

* Revert change

* Replace a couple more uri->did conversions

* Only filter replies from feeds if viewer hid them

* Revert, filter out replies that are hidden from feeds

* Remove old test

* Replace uri->did util

* Revert change to unused file

* Only validatePostEmbed and check postgates for post records

* Ensure notifications aren't generated down a hidden reply chain

* Changeset

* Cleanup

* Fix notification filtering logic

* Simplify

* Don't notify for invalid embeds

* Use new APIs

* Add hasPostGate and hasThreadGate flags from dataplane

* Only fetch postgates if post has one

* Only fetch threadgates if post has one or was deleted

* Remove notification filtering

* Don't hydrate threadgates for notifications

* Move hidden replies in feeds to match block handling

* Do no filtering of hidden replies in feeds

* Revert "Don't hydrate threadgates for notifications"

This reverts commit 1dcec0b239a7b9d6800427b26b8ba3e6a54210f9.

* Revert "Remove notification filtering"

This reverts commit 1e7069dfd809d1f18e9f05fd1d422e7399aa1bb0.

* Filter notifications for OP only

* Add additional check to hidden replies test

* Move noty filter logic into method handler

* Update .changeset/perfect-parrots-appear.md

Co-authored-by: devin ivy <devinivy@gmail.com>

* Update packages/bsky/tests/seed/postgates.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Another structuredClone

* Update packages/bsky/src/hydration/hydrator.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Better comment

* Update packages/bsky/src/data-plane/server/indexing/plugins/post.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Regen protos to match dataplane

* Update quotes snap to include embeddingDisabled

* Clarify usage of post uri -> gate utils

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2024-08-21 14:36:51 -05:00
Hailey
2a0c088cc5
quote aggs, list quotes (#2658)
* add quote count to post_agg, add getPostQuotes

rework schema

rework schema

add getPostQuotes to api

use posts

use posts

codegen

use items instead of quotes

codegen

add getPostQuotes

add quoteCount to response

update lexicon for postview

increment post ags

add quote to post aggs

add quote interface

oops

add quote table migration

* update

* bufgen

* update params

* update to use v2

* logs

* rm comment

* pass cursor

* add index

* Update packages/bsky/src/data-plane/server/db/migrations/20240723T220703655Z-quotes.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* only if its a post

* tests

* Discard changes to packages/bsky/tests/views/posts.test.ts

* fix client call

* Include new quotes agg in test expectation

* Use new API for headers

* Update packages/bsky/src/data-plane/server/indexing/plugins/post.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* revert rm

* rm timeout

* cursor test

* Changeset

* Remove pds specific bump

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
Co-authored-by: Eric Bailey <git@esb.lol>
2024-08-21 12:33:05 -05:00
github-actions[bot]
40c145fb16
Version packages (#2712)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-18 16:34:50 -04:00
Daniel Holmgren
50c0ec176c
Service auth method binding (lxm) (#2663)
* add scopes to service auth impl

* add error to getServiceAuth

* send scoped tokens from pds

* clean up privileged access scopes & allow simple service auth tokens for app passwords

* integration into ozone

* fix up bsky tests

* cleanup xrpc-server tests

* fix up tests & types

* one more test

* fix read after write tests

* fix mod auth test

* convert scopes to be a single method name

* add scope check callback for auth verifier

* pds changes only

* fix feed generation tests

* use scope for ozone service profile

* dont verify scopes on pds yet

* tidy

* tidy imports

* changeset

* add tests

* tidy

* another changeset

* scope -> lxm

* tidy

* clean up scope references

* update nonce size

* pr feedback

* trim trailing slash

* nonce -> jti

* fix xrpc-server test

* allow service auth on uploadBlob

* fix build error

* changeset

* build, tidy

* xrpc-server: update lxm claim check error

* appview: temporarily permit labeler service calls to omit lxm claim

* xrpc-server: fix test

* changeset

* fix merged tests

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2024-08-18 15:46:07 -04:00
Foysal Ahamed
bc131ebb01
Add index on createdAt column on moderation_event table (#2716)
*  Add index on createdAt column on moderation_event table

* Execute dropIndex

Co-authored-by: devin ivy <devinivy@gmail.com>

* tidy

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2024-08-15 18:25:31 -04:00
github-actions[bot]
cf988a5b4c
Version packages (#2709)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-12 22:55:19 -04:00
github-actions[bot]
3940733bf0
Version packages (#2706)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-12 16:11:19 -04:00
Matthieu Sieben
b934b396b1
Client SDK rework (#2483)
* feat(api): support creation of oauth based AtpAgents

* oauth: misc fixes for confidential clients

* fix(xprc): remove ReadableStream.from polyfill

* OAuth docs tweaks (#2679)

* OAuth: clarification about client_name being shown

* OAuth: re-write handle resolution privacy concern

* avoid relying on ReadableStream.from in xrpc-server tests

* feat(oauth-types): expose "ALLOW_UNSECURE_ORIGINS" constant

* feat(handle-resolver): expose "AtprotoIdentityDidMethods" type

* fix(oauth-client): ensure that the oauth metadata document contains client_id_metadata_document_supported

* fix(oauth-types): prevent unknown query string in loopback client id

* fix(identity-resolver): check that handle is in did doc's "alsoKnownAs"

* feat(oauth-client:oauth-resolver): allow logging in using either the PDS URL or Entryway URL

* fix(oauth-client): return better error in case of invalid "oauth-protected-resource" status code

* refactor(did): group atproto specific checks in own

* feat(api): relax typing of "appLabelers" and "labelers" AtpClient properties

* allow any did as labeller (for tests mainly)

* fix(api): allow to override "atproto-proxy" on a per-request basis

* remove release candidate versions from changelog

* update changeset for api and xrpc packages

* Add missing changeset

* revert RC versions

* Proper wording in OAUTH.md api example

* remove "pre" changeset file

* xrpc: restore original behavior of setHEader and unsetHeader

* docs: add comment for XrpcClient 's constructor arg

* feat(api): expose "schemas" publicly

* feat(api): allow customizing the whatwg fetch function of the AtpAgent

* docs(api): improve migration docs

* docs: change reference to BskyAgent to AtpAgent

* docs: mention the breaking change regarding setSessionPersistHandler

* fix(api): better split AtpClient concerns

* fix(xrpc): remove unused import

* refactor(api): simplify class hierarchu by removeing AtpClient

* fix(api): mock proper method for facets detection

* restore ability to restore session asynchronously

* feat(api): allow instantiating Agent with same argument as super class

* docs(api): properly extend Agent class

* style(xrpc): var name

* docs(api): remove "async" to header getter

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Hailey <me@haileyok.com>
2024-08-12 19:57:21 +02:00
github-actions[bot]
b0a5fa3d70
Version packages (#2685)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-05 15:21:57 -05:00
Daniel Holmgren
dc471da267
Service auth method binding - PDS (#2668)
* pds changes only

* use scope for ozone service profile

* dont verify scopes on pds yet

* tidy

* tidy imports

* changeset

* add tests

* another changeset

* scope -> lxm

* tidy

* update nonce size

* pr feedback

* trim trailing slash

* nonce -> jti

* fix xrpc-server test

* allow service auth on uploadBlob
2024-08-05 15:09:50 -05:00
bnewbold
a95a902bba
minor typos in descriptions and comments (#2681)
* lex: typos in descriptions

* more minor typos

* codegen lexicon typos

* more comment typos
2024-08-05 09:49:25 -07:00
github-actions[bot]
c007e930ed
Version packages (#2677)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-08-02 17:17:16 -05:00
Eric Bailey
951a3df15a
Remove detach (#2676)
* Remove detach record

* Changeset
2024-08-02 15:19:29 -05:00
github-actions[bot]
c5c5203a6e
Version packages (#2671)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-07-31 16:51:04 -05:00