Commit Graph

105 Commits

Author SHA1 Message Date
Matthieu Sieben d0c136cba2 Move PDS code over to @atproto/lex (#4408) 2026-03-23 18:10:16 +01:00
Foysal Ahamed 9f9f71a6a3 Add purge event for age assurance data for compliance (#4709)
*  Add purge event for age assurance data for compliance

* 🧹 Cleanup

* 📝 Add changeset

* :rotating_lights: Fix lint issue
2026-03-12 16:02:38 +01:00
rafael 7444295e79 use feature gate to remove image format from URL (#4721)
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 14:05:21 -03:00
devin ivy 383e157021 bsky: remove format from img urls by default (#4712)
* bsky: remove format from img urls, w/ config to control rollout. default webp.

* tidy

* changeset

* tidy
2026-03-04 22:41:44 -05:00
Geoff Cramer 95e377504a Create index for fetching mod events by creator (#4630)
* db: create index for moderation_event.createdBy

* tests: add queryEvents by creator test case

* refactor: add additional columns to createdBy index to match runtime query

* refactor: remove id since including both createdAt and id provide little incremental efficiency

* refactor: adding id back since it could matter in some cases
2026-02-11 14:52:46 -05:00
Emelia Smith 9fdfb8a3ad Prevent verification of handle.invalid (#4608) 2026-02-06 14:21:29 +01:00
Matthieu Sieben d54d7077eb DID document validation (#4580)
* DID document validation

* relax `import/no-deprecated` lint rule
2026-01-27 08:25:13 +01:00
Matthieu Sieben 261968fd65 New TS SDK (#4366)
* lex

* packaging

* moke packaging

* revert test changes

* do not build temp

* tidy

* automatically build the list of `@atproto/lex/com` lexicons

* fix build

* Remove "com" export

* ridy

* remove manifest option

* tidy

* rename

* tidy

* tidy

* tidy

* tests

* add procedure params

* stricter tests

* tidy

* Improve ui8 parsing

* tidy

* tidy

* code split

* code split

* fix reserved keywords conflict

* exclude packages/lex/src/tests/lexicons from lint

* reserved keywords

* safe identifier

* fix build

* move lib.js to src/lib.ts

* Move tests dir

* fix ci ?

* increast lint size

* Remove `Record` type alias for recordsz

* fix package json exports

* Add support for unsafe characters in defs and nsids

* tidy

* token tests

* tidy

* name consistency

* remove unused `unknownKeys` params option

* Fix "moving" keys in `DictSchema` (remove `IntersectionSchema`)

* REview comments

* adapt shebang in `env`

* Make sure union object have their $type property set in typings

* fix

* Improve typing of `UnknownTypedObject`

* lex improvements

* code reorg

* split lex-builder

* tidy

* improve packaging

* rename lex-validation to lex-schema

* lex client

* rename prettifier option

* add lex-client as dependency to "lex"

* Export client as part of main export

* re-write example app using @atproto/lex

* add missing lex-client to tsconfig

* tidy

* add "null" schema type

* Smaller bundle code footprint

* tidy

* correctness

* tidy

* code split and improved testing

* tidy

* refactor common utils

* test all implementations

* improve tests

* tidy

* fix build

* fixes

* tidy

* lint

* tests

* tidy

* fix oauth-example app

* tidy

* tidy

* tests

* tidy

* Return an actual `Uint8Array` from `fromBase64Node`

* tidy

* adapt xrpc-server

* Rename `Lex` to `LexValue`

* minor fixes

* fix tests

* fix tests

* tidy

* fix

* tidy

* tidy

* fix `verifyCidForBytes` implementation

* fix imports

* tidy

* split lex-json in own package

* make base64 tests faster

* Add interop tests

* lint error

* tidy

* tidy

* changeset

* implement lex-resolver and lex-install

* remove need for polyfill

* readme

* more details

* tidy

* allow specifying `service` header on a per request basis

* tidy

* tidy

* tidy

* tidy

* add custom/intersection validation schemas

* tidy

* tidy

* remive un-necessary util

* improve typing of `l.object` output

* make "name" required in lexicon method errors

* fix tests

* tidy

* tidy

* add error responses

* update readme

* add "like " to example

* readme improvements

* tidy

* error management improvements

* Improve error results

* tidy

* refactor

* tidy

* lock

* Update binary to `ts-lex`

* tidy

* tidy

* Add "Overview" section

* fix build

* update bin

* readme-improvements

* paul's feedback

* Update packages/lex/lex/README.md

Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>

* Update packages/lex/lex/README.md

Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>

* Update packages/lex/lex/README.md

Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>

* Update packages/lex/lex/README.md

Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>

* Update packages/lex/lex/README.md

Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>

* Initial plan

* Address README review comments

Co-authored-by: matthieusieben <813661+matthieusieben@users.noreply.github.com>

* Clarify client configuration inheritance behavior

Co-authored-by: matthieusieben <813661+matthieusieben@users.noreply.github.com>

* Document allowLegacyBlobs default and compatibility implications (#15)

* Initial plan

* Add notes about default setting and compatibility for allowLegacyBlobs

Co-authored-by: matthieusieben <813661+matthieusieben@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: matthieusieben <813661+matthieusieben@users.noreply.github.com>

* Rename `Json` to `JsonValue` in lex-json package (#14)

* Initial plan

* Rename Json to JsonValue in lex-json package and dependent packages

Co-authored-by: matthieusieben <813661+matthieusieben@users.noreply.github.com>

* Remove import alias for JsonValue in ipld.ts

Co-authored-by: matthieusieben <813661+matthieusieben@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: matthieusieben <813661+matthieusieben@users.noreply.github.com>

* Update packages/lex/lex/README.md

Co-authored-by: Paul Frazee <pfrazee@gmail.com>

* Update packages/lex/lex/README.md

Co-authored-by: Paul Frazee <pfrazee@gmail.com>

* Update packages/lex/lex/README.md

Co-authored-by: Paul Frazee <pfrazee@gmail.com>

* Update packages/lex/lex/README.md

Co-authored-by: Paul Frazee <pfrazee@gmail.com>

* Update packages/lex/lex/README.md

Co-authored-by: Paul Frazee <pfrazee@gmail.com>

* Update packages/lex/lex/README.md

Co-authored-by: Paul Frazee <pfrazee@gmail.com>

* review comments and fixes

* Add lex to the dockerfiles

* tidy

* changeset for lex packages

* tidy

* Tidy

* tidy

* tidy

* Move language parsing to lex-data

* tidy

* doctoc

* error handling

* tidy

* tidy

* tidy

* fix

---------

Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: matthieusieben <813661+matthieusieben@users.noreply.github.com>
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
2025-11-24 21:01:16 +01:00
Foysal Ahamed 032abf6b50 Update ozone action scheduler to queue emailing (#4340)
*  Update ozone action scheduler to queue emailing

* 🧹 Cleanup

* 📝 Add changeset
2025-11-19 10:26:14 +01:00
Daniel Holmgren 1dd20d3a81 WebSocket Library (#4348)
* create new websocket lib

* switch out ozone impl

* keepalive test

* remove websocket keepalive code from xrpc-server

* add the package in all the spots

* websocket -> ws-client

* missed ref

* fix import in ozone test
2025-11-18 17:34:41 -06:00
Foysal Ahamed 9115325c7b ❇️ Ozone takedown target service (#4344)
*  Add targetServices to takedown action to drive pds vs. appview takedowns

* 📝 Add changeset
2025-11-18 21:26:33 +01:00
Foysal Ahamed 601401afce Add account level strike count in ozone (#4279)
*  Add account level strike count in ozone

*  Update test snapshot

* 📝 Update lexicons

* 🧹 Cleanup

*  Allow objects in policy-sev level association

*  Allow negative strikeCount

* 📝 Add changeset
2025-10-27 16:45:22 +01:00
Foysal Ahamed ca768fe1b0 🐛 Fix query for ozone subject status with blobs (#4274)
* 🐛 Fix query for ozone subject status with blobs

* 📝 Add changeset
2025-10-14 09:31:58 +02:00
Foysal Ahamed 9c6b711c2e Remove flaky test since we already have coverage for randomization (#4260) 2025-10-06 19:05:47 +02:00
Foysal Ahamed 591de19524 Ozone delayed action (#4241)
*  Add delayed takedown feature to ozone

*  Apply tag auth check and add takedown event push

*  Add tests for takedown restrictions

* 🐛 Remove only test block

* 🐛 Add service forwarding

*  Copy event properties into scheduled takedown event

* 🧹 Cleanup

* 🐛 Make statuses required in listScheduledActions

* 🐛 Define required field in lexicon

*  Add tag when scheduling and cancelling scheduled takedown
2025-10-04 12:56:55 +02:00
Foysal Ahamed 55cc15cdd6 Add ozone proxy for revokeCredentials endpoint (#4170)
*  Add ozone proxy for revokeCredentials endpoint

* 📝 Add changeset

*  Add mod event for revoke credentials

*  Add tests
2025-09-09 14:13:48 +02:00
Eric Bailey a5b20f0218 Add expanded moderation report reasons (#3881)
* Integrate new reporting reasons

* Update bnn to BNR, prefix all with reason* to match previous

* Remap deprecations

* Update naming, add notes about Bluesky-only reasons

* Update reason

* Move new defs to tools.ozone namespace

* Add ozone lexicons to app view

* Copy known values to merge defs

* Update comments

* Add reasonAppeal to new ozone namespace defs

* Changeset

* ❇️ Support new reporting categories in ozone (#3974)

*  Validate report reason using labeler service profile

*  Rename test

* :rotating_lights: Fix lint issue

*  Use both appeal reason type for materialized views

*  Add old to new reason mapping for fallback

*  Update test snapshot

* :rotating_lights: Fix lint issue

* 🧹 Cleanup

* :rotating_lights: Fix lint issue

*  Adjust report reason tagging

* 📝 Additional comment for new migration

---------

Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
2025-09-02 21:40:31 +02:00
Foysal Ahamed 3156ddf615 Add batchId filter for queryEvents (#4109)
*  Add batchId filter for queryEvents

* 📝 Add changeset
2025-08-20 21:39:24 +02:00
Foysal Ahamed 171efadb49 ❇️ Ozone events timeline (#3927)
* 🚧 WIP

*  Blend in plc and account history events in account timeline

*  Adjust snapshot

* 🧹 Cleanup and address review comments

*  Update test snapshot

* 🐛 Add back the helper function

* 📝 Add explainer comment

*  Map plc events to conforming naming convention

*  Update test snapshot

* 🧹 Cleanup

*  Add known values for event types

* 📝 Add changeset
2025-08-04 20:37:05 +02:00
rafael de29a346d8 Fix paginateAll mistakes and improve typing (#4062) 2025-07-22 14:51:16 -03:00
Foysal Ahamed 3b356c5096 External ID in ozone events to help dedupe events (#4048)
*  Add externalId to ozone events for deduping events per subject and event type

* 📝 Add changeset

*  Move duplicate event check inside transaction
2025-07-15 18:32:22 +02:00
Foysal Ahamed 926193f825 Ozone support for age assurance compliance events (#4029)
* 🚧 WIP

*  Make age assurance state queryable

*  Split age assurance events into 2

*  Implement admin and user state overrides

*  Add blocked as a known value for age assurance state

*  Update test snapshot

*  Update test snapshot

*  Cleanup
2025-07-11 18:00:55 +02:00
Foysal Ahamed 0c0381a2bb ❇️ Ozone user agent (#3991)
*  Add userAgent tracking for events

* :rotating_lights: Fix lint issue

* ♻️ Refactor userAgent to modTool

*  Rename extra to meta

* 📝 Add changeset

*  Support modTools param in createReport

*  Add support for mod tool in createReport
2025-07-02 22:34:15 +02:00
Foysal Ahamed 02c358d0ca Adds safelink module (#3945)
*  Adds safelink module

*  Remove createdAt timestamp bloat

* :rotating_lights: Fix lint issue

* 🐛 Fix pagination

* 🔨 Refactor safelink rule table

* 🧹 Add better default

*  Better search params

*  Remove mod requirement for query rules and events

*  Cleanup search for queryEvents

* 📝 Add changeset

* :rotating_lights: Fix lint issue

* 🧹 Adjust as per review feedback

*  Add support for sort direction in safelink rules

* :rotating_lights: Fix lint issue

*  Split input and response object shape

* :rotating_lights: Fix lint issue
2025-07-02 21:17:38 +02:00
Samuel Newman 8a0f190399 Subscribe to activity implementation (#3908)
Co-authored-by: rafael <rafael@blueskyweb.xyz>
2025-06-30 18:43:27 -03:00
Foysal Ahamed 498f3c0738 🐛 Fetch record from pds if appview fails to find it for ozone (#3892)
* 🐛 Fetch record from pds if appview fails to find it for ozone

*  Resolve and etch from pds without auth

* ♻️ Refactor and cleanup

*  Fix tests

*  Fix tests

* 🚨 Fix linter issue

* 🧹 Cleanup
2025-05-28 21:39:28 +02:00
Foysal Ahamed 5fd7ef0bf6 🐛 Ignore expired labels when queried from ozone database (#3786)
* 🐛 Ignore expired labels when queried from ozone database

* 🚨 remove unused imports
2025-04-30 17:58:08 +02:00
Foysal Ahamed 3301a2697f 🐛 Verification cleanups (#3802)
* 🐛 Remove default jetstream url in ozone

*  Prevent publishing multiple verification records

* 📝 Add changeset
2025-04-28 22:51:09 +02:00
Foysal Ahamed 45354c84f8 ❇️ Verifications manager in ozone (#3765)
* First vouch implementation

* Remove unneeded endpoints

* wip

*  wip

*  Process jetstream events through p-queue and add tests

*  Add test for cursor update

* 🐛 Use utc time to update updatedAt

* 🧹 Cleanup

* 🔨 Fix pnpm versioning issues

*  Replace jetstream lib with manual implementation

*  Remove unnecessary 3p dep

*  Add e2e test for jetstream

* 🚨 Fix import

* 🧹 Remove unnecessary property

*  Fix dev-env and add profile to verification view in ozone

*  Add profile type

*  Add backpressure handling to jetstream listener

*  Use WebSocketKeepAlive from xrpc-server and replace partysocket

*  Add a new verifier role to ozone team meber roles

* 📝 Run codegen

* 🐛 Fix auth check

* 🐛 Fix test failure check

* 🚨 Fix json formatting

* 🐛 Fix team role check

* 🚧 Checking failing test

*  Fix tests

*  Address review comments

*  Add xrpc-server to version

* 🚨 Fix linter issue

* 🚨 Fix linter issue

*  Resolve race condition in cursor update

*  Add verification check on profile

* 🐛 Fix missing cid in test and firehose cursor

*  Fix test

*  Add record validation for verification and separate xrpc-server version

*  Return error object for failed revocations

*  Add re-login on expired session case

* 📝 Fix typo

---------

Co-authored-by: rafael <rafael@blueskyweb.xyz>
2025-04-24 10:49:45 +02:00
devin ivy 9a585b6a2c Ozone: fix correctness and perf issues with reporter stats (#3791)
ozone: fix correctness and perf issues with reporter stats
2025-04-22 10:13:45 -04:00
Foysal Ahamed ac6b604e7a 🐛 Pass auth headers when fetching profiles for ozone subjects (#3683)
* 🐛 Pass auth headers when fetching profiles for ozone subjects

*  Update snapshot
2025-03-27 17:07:05 -04:00
Foysal Ahamed 076c2f9872 Add getSubjects endpoint to ozone for full view of subjects (#3651)
*  Add getSubjects endpoint to ozone for full view of subjects

* 📝 Better lexicon description

* 📝 Add changeset

*  Add subject to subject view

*  Update snapshot

* :rotating_lights: Fix lint issue

*  Make subject required in subjectView

* 🐛 Add subject check before attempting to run queries

*  Keed the order of subjects from the request in the responding array

*  Use a union for profile field

* 🐛 Remove minLength and address profile value

* 📝 Bring back minLength
2025-03-27 10:50:44 -04:00
Foysal Ahamed 18fbfa0005 Add team member handle and displayName (#3587)
*  Add team member handle and displayName

*  Add periodic profile details sync

* 📝 Add changeset

*  Update team service instantiation

* 🐛 Cleanup

* 🧹 Cleanup unnecessary index
2025-03-07 23:00:25 +00:00
Foysal Ahamed 7449f8607c Allow filtering ozone team list by role and status (#3554)
*  Allow filtering ozone team list by role and status

* fix style

* changeset

---------

Co-authored-by: Matthieu Sieben <matthieu.sieben@gmail.com>
2025-02-19 10:15:58 +01:00
Foysal Ahamed b41ff4b4e3 Add reporter_stats materialized view and endpoint to fetch reporter stats (#3509)
*  Add reporter_stats materialized view and endpoint to fetch reporter stats

* 🚨 Fix linter issues

*  Change reporter stats query from materialized view to on demand select

* Add "createdAt" as part of the index

---------

Co-authored-by: Matthieu Sieben <matthieu.sieben@gmail.com>
Co-authored-by: Matthieu Sieben <matthieusieben@users.noreply.github.com>
2025-02-17 22:17:43 +01:00
Matthieu Sieben c53d943c8b Improve codegen typings (#2999)
* Make codegen types stricter
* Add .js file extension to import statements in generated code
* Fixes a bug that would clear interests prefs when updating hidden posts prefs.
2025-02-13 15:21:00 +01:00
Foysal Ahamed 709a85b0b6 Add a priority score value for ozone subjects (#3495)
*  Add a priority score value for ozone subjects

* 🚨 Bring back lock file

*  Update snapshot for pds proxied tests

*  Update takedown appeal snapshot

*  Update takedown appeal snapshot

*  Update repo and record snapshots

* :rotating_lights: Fix lint issue

*  Update moderation events snapshot

*  Update moderation statuses snapshot

* 🧹 Cleanup unnecessary durationInHours field

*  Allow filtering by priority score

*  Add test for priorityScore filter

* 🚨 Fix import order

*  Allow 0 priority score filter

* 📝 Add changeset
2025-02-06 20:54:27 +00:00
Matthieu Sieben 61dc0d60e1 Add linting rule to sort imports (#3220)
* Add linting rule to sort imports

* remove spacing between import groups

* changeset

* changeset

* prettier config fine tuning

* forbid use of deprecated imports

* tidy
2025-02-05 15:06:58 +01:00
Foysal Ahamed d377d1a9be Allow creating labels with expiry (#3476)
*  Allow creating labels with expiry

* 🧹 Cleanup
2025-01-31 11:48:10 -08:00
Matthieu Sieben 8810885b8e fix pds tests flakiness (#3463) 2025-01-26 13:24:03 -08:00
Matthieu Sieben ee9779d074 Add account details to queryStatuses (#3236)
* Moderation subjectStatusView with statistics

* add account stats to queue entries

* fix

* adapt

* review changes

* style

* review comments

* wip

* add indexes

* refresh materialized view

* use the background queue to refresh materialized views

* tidy

* fix verrify

* tidy

* Add filtering based on "minAccountSuspendCount"

* fix tests

* Update test snapshots

* update materialized views when running `processAll`

* tidy

* processAll sequentially

* tidy

* tidy

* fix dead lock
2025-01-20 15:36:02 +01:00
Matthieu Sieben 72eba67af1 Drop axios dependency (#3177)
* Minor adaptation of VerifyCidTransform implementation

* refactor: factorize content-encoding negotiation into new lib

* bsky: Use undici to stream blob

* fixup! bsky: Use undici to stream blob

* disable ssrf bsky protection in dev-env

* remove http requests to self to host "/img/"

* drop axios from tests

* fixes

* fix tests

* reviex changes

* properly handle HEAD requests

* handle client disconnection

* fix tests

* drop unrelated change

* tidy

* tidy

* tidy

* remove axios from dev-env

* remove axios from identity package

* use undici 6

* remove axios dependency from ozone

* tidy

* remove axios from PDS package

* avoid killing bsky-pds connections

* improve debugging data

* Better handle invalid CID

* tidy

* tidy

* refactor "allFulfilled" util in @atproto/common

* tidy

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2025-01-06 18:34:11 +01:00
Foysal Ahamed 53621f8e10 Add a policy property to takedown events (#3271)
*  Add a policy property to takedown events

*  Add policy list setting validation

*  Make multiple policies possible for takedown and event search

* 📝 Add changeset

*  Use , as policies separator
2025-01-02 20:46:53 -05:00
Foysal Ahamed c559e7e3de Add subject tags with report type (#3260)
*  Add subject tags with report type

*  Update snapshots

*  Update snpshot for pds admin

*  Update ozone snapshot

*  Update ozone snapshot

* 🧹 Cleanup
2024-12-18 22:57:27 +00:00
Foysal Ahamed 6a3e781b98 Add && separator for tags filter param on queryStatuses endpoint (#3261)
*  Add && separator for tags filter param on queryStatuses endpoint

* 🐛 Handle potential empty tag search input

* 🐛 Handle empty condition
2024-12-18 22:46:05 +00:00
Foysal Ahamed e9240c8b5e 🐛 Retrieve takendown profiles from app.bsky.actor.getProfiles based on auth (#3206) 2024-12-07 16:47:02 -05:00
Foysal Ahamed 32baddf4a9 🐛 Fix auth level check for protected tags (#3205) 2024-12-07 12:02:42 -05:00
Foysal Ahamed ed22362209 Optionally ack all open subjects from the author with acknowledge event (#3171)
*  Acknowledge all open subjects of an account when acknowledging account level reports

* 📝 Add changeset
2024-12-05 11:45:28 +00:00
Foysal Ahamed c72145dbeb Allow querying events by multiple keywords using OR condition (#3070)
*  Allow querying events by multiple keywords using OR condition

* 📝 Update comment

*  Update operator to ||

*  Fix test

* 🐛 Handle edge cases around search query

*  Codgen
2024-11-29 18:00:48 +00:00
Foysal Ahamed 011e73ec05 Add protected tag setting (#3050)
*  Add protected tag setting

*  Add tests for protected tag options

*  Validate mod and role list

* 🧹 Replace usage of objects with Map

* 🐛 Fix setting validator getter
2024-11-22 15:28:53 +00:00