90 Commits

Author SHA1 Message Date
Foysal Ahamed
55cc15cdd6
Add ozone proxy for revokeCredentials endpoint (#4170)
*  Add ozone proxy for revokeCredentials endpoint

* 📝 Add changeset

*  Add mod event for revoke credentials

*  Add tests
2025-09-09 14:13:48 +02:00
Eric Bailey
a5b20f0218
Add expanded moderation report reasons (#3881)
* Integrate new reporting reasons

* Update bnn to BNR, prefix all with reason* to match previous

* Remap deprecations

* Update naming, add notes about Bluesky-only reasons

* Update reason

* Move new defs to tools.ozone namespace

* Add ozone lexicons to app view

* Copy known values to merge defs

* Update comments

* Add reasonAppeal to new ozone namespace defs

* Changeset

* ❇️ Support new reporting categories in ozone (#3974)

*  Validate report reason using labeler service profile

*  Rename test

* :rotating_lights: Fix lint issue

*  Use both appeal reason type for materialized views

*  Add old to new reason mapping for fallback

*  Update test snapshot

* :rotating_lights: Fix lint issue

* 🧹 Cleanup

* :rotating_lights: Fix lint issue

*  Adjust report reason tagging

* 📝 Additional comment for new migration

---------

Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
2025-09-02 21:40:31 +02:00
Foysal Ahamed
3156ddf615
Add batchId filter for queryEvents (#4109)
*  Add batchId filter for queryEvents

* 📝 Add changeset
2025-08-20 21:39:24 +02:00
Foysal Ahamed
171efadb49
❇️ Ozone events timeline (#3927)
* 🚧 WIP

*  Blend in plc and account history events in account timeline

*  Adjust snapshot

* 🧹 Cleanup and address review comments

*  Update test snapshot

* 🐛 Add back the helper function

* 📝 Add explainer comment

*  Map plc events to conforming naming convention

*  Update test snapshot

* 🧹 Cleanup

*  Add known values for event types

* 📝 Add changeset
2025-08-04 20:37:05 +02:00
rafael
de29a346d8
Fix paginateAll mistakes and improve typing (#4062) 2025-07-22 14:51:16 -03:00
Foysal Ahamed
3b356c5096
External ID in ozone events to help dedupe events (#4048)
*  Add externalId to ozone events for deduping events per subject and event type

* 📝 Add changeset

*  Move duplicate event check inside transaction
2025-07-15 18:32:22 +02:00
Foysal Ahamed
926193f825
Ozone support for age assurance compliance events (#4029)
* 🚧 WIP

*  Make age assurance state queryable

*  Split age assurance events into 2

*  Implement admin and user state overrides

*  Add blocked as a known value for age assurance state

*  Update test snapshot

*  Update test snapshot

*  Cleanup
2025-07-11 18:00:55 +02:00
Foysal Ahamed
0c0381a2bb
❇️ Ozone user agent (#3991)
*  Add userAgent tracking for events

* :rotating_lights: Fix lint issue

* ♻️ Refactor userAgent to modTool

*  Rename extra to meta

* 📝 Add changeset

*  Support modTools param in createReport

*  Add support for mod tool in createReport
2025-07-02 22:34:15 +02:00
Foysal Ahamed
02c358d0ca
Adds safelink module (#3945)
*  Adds safelink module

*  Remove createdAt timestamp bloat

* :rotating_lights: Fix lint issue

* 🐛 Fix pagination

* 🔨 Refactor safelink rule table

* 🧹 Add better default

*  Better search params

*  Remove mod requirement for query rules and events

*  Cleanup search for queryEvents

* 📝 Add changeset

* :rotating_lights: Fix lint issue

* 🧹 Adjust as per review feedback

*  Add support for sort direction in safelink rules

* :rotating_lights: Fix lint issue

*  Split input and response object shape

* :rotating_lights: Fix lint issue
2025-07-02 21:17:38 +02:00
Samuel Newman
8a0f190399
Subscribe to activity implementation (#3908)
Co-authored-by: rafael <rafael@blueskyweb.xyz>
2025-06-30 18:43:27 -03:00
Foysal Ahamed
498f3c0738
🐛 Fetch record from pds if appview fails to find it for ozone (#3892)
* 🐛 Fetch record from pds if appview fails to find it for ozone

*  Resolve and etch from pds without auth

* ♻️ Refactor and cleanup

*  Fix tests

*  Fix tests

* 🚨 Fix linter issue

* 🧹 Cleanup
2025-05-28 21:39:28 +02:00
Foysal Ahamed
5fd7ef0bf6
🐛 Ignore expired labels when queried from ozone database (#3786)
* 🐛 Ignore expired labels when queried from ozone database

* 🚨 remove unused imports
2025-04-30 17:58:08 +02:00
Foysal Ahamed
3301a2697f
🐛 Verification cleanups (#3802)
* 🐛 Remove default jetstream url in ozone

*  Prevent publishing multiple verification records

* 📝 Add changeset
2025-04-28 22:51:09 +02:00
Foysal Ahamed
45354c84f8
❇️ Verifications manager in ozone (#3765)
* First vouch implementation

* Remove unneeded endpoints

* wip

*  wip

*  Process jetstream events through p-queue and add tests

*  Add test for cursor update

* 🐛 Use utc time to update updatedAt

* 🧹 Cleanup

* 🔨 Fix pnpm versioning issues

*  Replace jetstream lib with manual implementation

*  Remove unnecessary 3p dep

*  Add e2e test for jetstream

* 🚨 Fix import

* 🧹 Remove unnecessary property

*  Fix dev-env and add profile to verification view in ozone

*  Add profile type

*  Add backpressure handling to jetstream listener

*  Use WebSocketKeepAlive from xrpc-server and replace partysocket

*  Add a new verifier role to ozone team meber roles

* 📝 Run codegen

* 🐛 Fix auth check

* 🐛 Fix test failure check

* 🚨 Fix json formatting

* 🐛 Fix team role check

* 🚧 Checking failing test

*  Fix tests

*  Address review comments

*  Add xrpc-server to version

* 🚨 Fix linter issue

* 🚨 Fix linter issue

*  Resolve race condition in cursor update

*  Add verification check on profile

* 🐛 Fix missing cid in test and firehose cursor

*  Fix test

*  Add record validation for verification and separate xrpc-server version

*  Return error object for failed revocations

*  Add re-login on expired session case

* 📝 Fix typo

---------

Co-authored-by: rafael <rafael@blueskyweb.xyz>
2025-04-24 10:49:45 +02:00
devin ivy
9a585b6a2c
Ozone: fix correctness and perf issues with reporter stats (#3791)
ozone: fix correctness and perf issues with reporter stats
2025-04-22 10:13:45 -04:00
Foysal Ahamed
ac6b604e7a
🐛 Pass auth headers when fetching profiles for ozone subjects (#3683)
* 🐛 Pass auth headers when fetching profiles for ozone subjects

*  Update snapshot
2025-03-27 17:07:05 -04:00
Foysal Ahamed
076c2f9872
Add getSubjects endpoint to ozone for full view of subjects (#3651)
*  Add getSubjects endpoint to ozone for full view of subjects

* 📝 Better lexicon description

* 📝 Add changeset

*  Add subject to subject view

*  Update snapshot

* :rotating_lights: Fix lint issue

*  Make subject required in subjectView

* 🐛 Add subject check before attempting to run queries

*  Keed the order of subjects from the request in the responding array

*  Use a union for profile field

* 🐛 Remove minLength and address profile value

* 📝 Bring back minLength
2025-03-27 10:50:44 -04:00
Foysal Ahamed
18fbfa0005
Add team member handle and displayName (#3587)
*  Add team member handle and displayName

*  Add periodic profile details sync

* 📝 Add changeset

*  Update team service instantiation

* 🐛 Cleanup

* 🧹 Cleanup unnecessary index
2025-03-07 23:00:25 +00:00
Foysal Ahamed
7449f8607c
Allow filtering ozone team list by role and status (#3554)
*  Allow filtering ozone team list by role and status

* fix style

* changeset

---------

Co-authored-by: Matthieu Sieben <matthieu.sieben@gmail.com>
2025-02-19 10:15:58 +01:00
Foysal Ahamed
b41ff4b4e3
Add reporter_stats materialized view and endpoint to fetch reporter stats (#3509)
*  Add reporter_stats materialized view and endpoint to fetch reporter stats

* 🚨 Fix linter issues

*  Change reporter stats query from materialized view to on demand select

* Add "createdAt" as part of the index

---------

Co-authored-by: Matthieu Sieben <matthieu.sieben@gmail.com>
Co-authored-by: Matthieu Sieben <matthieusieben@users.noreply.github.com>
2025-02-17 22:17:43 +01:00
Matthieu Sieben
c53d943c8b
Improve codegen typings (#2999)
* Make codegen types stricter
* Add .js file extension to import statements in generated code
* Fixes a bug that would clear interests prefs when updating hidden posts prefs.
2025-02-13 15:21:00 +01:00
Foysal Ahamed
709a85b0b6
Add a priority score value for ozone subjects (#3495)
*  Add a priority score value for ozone subjects

* 🚨 Bring back lock file

*  Update snapshot for pds proxied tests

*  Update takedown appeal snapshot

*  Update takedown appeal snapshot

*  Update repo and record snapshots

* :rotating_lights: Fix lint issue

*  Update moderation events snapshot

*  Update moderation statuses snapshot

* 🧹 Cleanup unnecessary durationInHours field

*  Allow filtering by priority score

*  Add test for priorityScore filter

* 🚨 Fix import order

*  Allow 0 priority score filter

* 📝 Add changeset
2025-02-06 20:54:27 +00:00
Matthieu Sieben
61dc0d60e1
Add linting rule to sort imports (#3220)
* Add linting rule to sort imports

* remove spacing between import groups

* changeset

* changeset

* prettier config fine tuning

* forbid use of deprecated imports

* tidy
2025-02-05 15:06:58 +01:00
Foysal Ahamed
d377d1a9be
Allow creating labels with expiry (#3476)
*  Allow creating labels with expiry

* 🧹 Cleanup
2025-01-31 11:48:10 -08:00
Matthieu Sieben
8810885b8e
fix pds tests flakiness (#3463) 2025-01-26 13:24:03 -08:00
Matthieu Sieben
ee9779d074
Add account details to queryStatuses (#3236)
* Moderation subjectStatusView with statistics

* add account stats to queue entries

* fix

* adapt

* review changes

* style

* review comments

* wip

* add indexes

* refresh materialized view

* use the background queue to refresh materialized views

* tidy

* fix verrify

* tidy

* Add filtering based on "minAccountSuspendCount"

* fix tests

* Update test snapshots

* update materialized views when running `processAll`

* tidy

* processAll sequentially

* tidy

* tidy

* fix dead lock
2025-01-20 15:36:02 +01:00
Matthieu Sieben
72eba67af1
Drop axios dependency (#3177)
* Minor adaptation of VerifyCidTransform implementation

* refactor: factorize content-encoding negotiation into new lib

* bsky: Use undici to stream blob

* fixup! bsky: Use undici to stream blob

* disable ssrf bsky protection in dev-env

* remove http requests to self to host "/img/"

* drop axios from tests

* fixes

* fix tests

* reviex changes

* properly handle HEAD requests

* handle client disconnection

* fix tests

* drop unrelated change

* tidy

* tidy

* tidy

* remove axios from dev-env

* remove axios from identity package

* use undici 6

* remove axios dependency from ozone

* tidy

* remove axios from PDS package

* avoid killing bsky-pds connections

* improve debugging data

* Better handle invalid CID

* tidy

* tidy

* refactor "allFulfilled" util in @atproto/common

* tidy

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2025-01-06 18:34:11 +01:00
Foysal Ahamed
53621f8e10
Add a policy property to takedown events (#3271)
*  Add a policy property to takedown events

*  Add policy list setting validation

*  Make multiple policies possible for takedown and event search

* 📝 Add changeset

*  Use , as policies separator
2025-01-02 20:46:53 -05:00
Foysal Ahamed
c559e7e3de
Add subject tags with report type (#3260)
*  Add subject tags with report type

*  Update snapshots

*  Update snpshot for pds admin

*  Update ozone snapshot

*  Update ozone snapshot

* 🧹 Cleanup
2024-12-18 22:57:27 +00:00
Foysal Ahamed
6a3e781b98
Add && separator for tags filter param on queryStatuses endpoint (#3261)
*  Add && separator for tags filter param on queryStatuses endpoint

* 🐛 Handle potential empty tag search input

* 🐛 Handle empty condition
2024-12-18 22:46:05 +00:00
Foysal Ahamed
e9240c8b5e
🐛 Retrieve takendown profiles from app.bsky.actor.getProfiles based on auth (#3206) 2024-12-07 16:47:02 -05:00
Foysal Ahamed
32baddf4a9
🐛 Fix auth level check for protected tags (#3205) 2024-12-07 12:02:42 -05:00
Foysal Ahamed
ed22362209
Optionally ack all open subjects from the author with acknowledge event (#3171)
*  Acknowledge all open subjects of an account when acknowledging account level reports

* 📝 Add changeset
2024-12-05 11:45:28 +00:00
Foysal Ahamed
c72145dbeb
Allow querying events by multiple keywords using OR condition (#3070)
*  Allow querying events by multiple keywords using OR condition

* 📝 Update comment

*  Update operator to ||

*  Fix test

* 🐛 Handle edge cases around search query

*  Codgen
2024-11-29 18:00:48 +00:00
Foysal Ahamed
011e73ec05
Add protected tag setting (#3050)
*  Add protected tag setting

*  Add tests for protected tag options

*  Validate mod and role list

* 🧹 Replace usage of objects with Map

* 🐛 Fix setting validator getter
2024-11-22 15:28:53 +00:00
Matthieu Sieben
c34426fc55
Fix ozone preferences test (#3001)
use consistent ordering of insertion
2024-11-14 14:27:49 +01:00
Matthieu Sieben
879a74f7b8
Fix failing CI (#3000)
* fix

* use distinct pg schemas for all tests
2024-11-14 14:13:58 +01:00
Foysal Ahamed
d6f33b4742
Add events for account and record update/delete/deactivation (#2661)
*  Add events for account and record update/delete/deactivation

*  Add handle change event

*  Reduce account events to 2 types and record events to 1

*  Store metadata from account, identity and record events

*  Add created event for record

*  Add ndd the new events to allowed types in emitEvent

*  Use string value for record op and add tombstone flag to identity event

*  Add active flag on account events

*  Change accountStatus -> status to match with firehose event

*  Make active flag required

* 🚨 fix prettier style issue

*  Track record/account delete and update data in subject status (#2804)

*  Store deleted/updated event data in subject_status

* 🐛 Fix query for recordDeletedAt and recordUpdatedAt

*  Add tombstoned status

*  Move from record to hosting term

*  Add tests for hosting params

*  Update lexicons for hostingStatuses

*  Update snapshots

*  Update snapshots

*  Update snapshots

*  Adjust hosting statuses

* 📝 Add changeset
2024-11-07 22:51:44 +01:00
Foysal Ahamed
c4b5e53957
Ozone instance-wide and user-specific settings (#2905)
*  Settings endpoints are working

* 🧹 Rename file

*  Replace ad-hoc manage roles to match team member roles

* ♻️ Refactor role names

*  Polish up

*  Move to using id for pagination

* 📝 Add changeset

*  Update snapshots

*  Change column order in setting table index and add did in all queries
2024-11-07 22:43:30 +01:00
Foysal Ahamed
19e36afb2c
Add collection and subjectType filters to queryEvents and queryStatuses (#2914)
*  Add collection and subjectType filters to queryEvents and queryStatuses

* 📝 Add changeset

* ♻️ Refactor or query builder
2024-10-30 16:20:38 +01:00
Foysal Ahamed
22d039a229
Ozone sets (#2636)
*  Initial implementation of sets api on ozone

*  Introduce sortDirection to querySets

* 🧹 Cleanup and refactor

*  Align setView for response

* ♻️ Rename and add specific error

* 🐛 Cleanup unnecessary check that is covered by lexicon

*  Rename remove to delete and add set suffix

*  Use id and createdAt for values pagination

*  Add index on createdAt for query perf and other cleanups

* 🐛 Set createdAt when inserting values

* 📝 Add changeset

*  Add index on setId and createdAt
2024-10-08 19:16:09 +02:00
Foysal Ahamed
a2bad977a8
Ozone batch repo and record getters (#2836)
*  Add getRepos and getRecords endpoints for bulk fetching

*  Fix issues and add tests for get repos and get records

*  Use the right lxm

* 🐛 Revert changes in lockfile

*  Add getAccountInfos in PDS

* 🐛 Fix type def for repo and record view detail

*  Update snapshots

*  Update snapshots

*  Consolidate error type for com.atproto and tools.ozone getRecord error type

* 🧹 Cleanup

*  Update snapshots

*  Update snapshots

*  Changeset
2024-10-01 19:37:23 +02:00
Foysal Ahamed
62a8225f36
Optionally ack all open subjects from the author with takedown event and get all mod subjects of a user (#2793)
*  Add acknowledgeAllSubjectsOfAccount flag with takedown event

* 📝 better documentation of forAccount and subject params

* 🧹 Cleanup tests

*  Change wording according to review

*  Refactor tests

* ♻️ Rename acknowledge flag
2024-09-11 18:26:53 +02:00
Foysal Ahamed
372ed4c519
Apply embed specific tag on subjects for video, image and external (#2703)
*  Refactor subject tagging to facilitate video content tagging

* ♻️ Refactor tag check

*  Fix tagging logic

* ♻️ Refactor content tagger and fix image content type check

*  Add embed tag check for video and external

*  Add tagging for both media and image embed
2024-08-29 18:23:48 +02:00
Daniel Holmgren
50c0ec176c
Service auth method binding (lxm) (#2663)
* add scopes to service auth impl

* add error to getServiceAuth

* send scoped tokens from pds

* clean up privileged access scopes & allow simple service auth tokens for app passwords

* integration into ozone

* fix up bsky tests

* cleanup xrpc-server tests

* fix up tests & types

* one more test

* fix read after write tests

* fix mod auth test

* convert scopes to be a single method name

* add scope check callback for auth verifier

* pds changes only

* fix feed generation tests

* use scope for ozone service profile

* dont verify scopes on pds yet

* tidy

* tidy imports

* changeset

* add tests

* tidy

* another changeset

* scope -> lxm

* tidy

* clean up scope references

* update nonce size

* pr feedback

* trim trailing slash

* nonce -> jti

* fix xrpc-server test

* allow service auth on uploadBlob

* fix build error

* changeset

* build, tidy

* xrpc-server: update lxm claim check error

* appview: temporarily permit labeler service calls to omit lxm claim

* xrpc-server: fix test

* changeset

* fix merged tests

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2024-08-18 15:46:07 -04:00
Matthieu Sieben
b934b396b1
Client SDK rework (#2483)
* feat(api): support creation of oauth based AtpAgents

* oauth: misc fixes for confidential clients

* fix(xprc): remove ReadableStream.from polyfill

* OAuth docs tweaks (#2679)

* OAuth: clarification about client_name being shown

* OAuth: re-write handle resolution privacy concern

* avoid relying on ReadableStream.from in xrpc-server tests

* feat(oauth-types): expose "ALLOW_UNSECURE_ORIGINS" constant

* feat(handle-resolver): expose "AtprotoIdentityDidMethods" type

* fix(oauth-client): ensure that the oauth metadata document contains client_id_metadata_document_supported

* fix(oauth-types): prevent unknown query string in loopback client id

* fix(identity-resolver): check that handle is in did doc's "alsoKnownAs"

* feat(oauth-client:oauth-resolver): allow logging in using either the PDS URL or Entryway URL

* fix(oauth-client): return better error in case of invalid "oauth-protected-resource" status code

* refactor(did): group atproto specific checks in own

* feat(api): relax typing of "appLabelers" and "labelers" AtpClient properties

* allow any did as labeller (for tests mainly)

* fix(api): allow to override "atproto-proxy" on a per-request basis

* remove release candidate versions from changelog

* update changeset for api and xrpc packages

* Add missing changeset

* revert RC versions

* Proper wording in OAUTH.md api example

* remove "pre" changeset file

* xrpc: restore original behavior of setHEader and unsetHeader

* docs: add comment for XrpcClient 's constructor arg

* feat(api): expose "schemas" publicly

* feat(api): allow customizing the whatwg fetch function of the AtpAgent

* docs(api): improve migration docs

* docs: change reference to BskyAgent to AtpAgent

* docs: mention the breaking change regarding setSessionPersistHandler

* fix(api): better split AtpClient concerns

* fix(xrpc): remove unused import

* refactor(api): simplify class hierarchu by removeing AtpClient

* fix(api): mock proper method for facets detection

* restore ability to restore session asynchronously

* feat(api): allow instantiating Agent with same argument as super class

* docs(api): properly extend Agent class

* style(xrpc): var name

* docs(api): remove "async" to header getter

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: bnewbold <bnewbold@robocracy.org>
Co-authored-by: Hailey <me@haileyok.com>
2024-08-12 19:57:21 +02:00
devin ivy
f05539db4c
Appview: apply hosting status in getRecord (#2620)
appview: apply hosting status in getRecord
2024-07-02 17:43:21 -04:00
devin ivy
af7d3912a3
Starter packs (#2553)
* lexicon: initial starter pack record lexicons

* lexicon: first pass on views for starter packs

* lexicon: add starter pack query endpoints, misc fixes

* lexicon: add starter pack related fields to profile views

* lexicon: codegen

* api prerelease

* bsky: protos for starter packs

* appview: implement starter pack endpoints in dataplane, protos update

* appview: starter pack hydration and view, mock dataplane indexing

* appview: wire-up starter pack graph endpoints

* appview: test basic starter packs functionality, update dev-env

* fix snaps for profile createdAt

* appview: fix list item count for starter packs in dataplane

* fix ozone snaps, tidy

* appview: reorg list aggs for starter packs

* appview: starter pack lexicon and proto tweaks, move around aggregations

* appview: fix views and snaps

* api prerelease

* fix snaps

* appview: misc proto updates for starter packs, snaps

* appview: add starter pack counts to profile detail

* build

* api prerelease

* pds: starter pack slur detection

* lexicon: starter pack notifications and batch fetch

* appview: starter pack notification and batch fetch

* appview: add starterpack-joined notif to dataplane and test

* api prerelease

* ozone: fix snaps

* pds build

*  Proxy starterpack requests through ozone and include takedowns (#2594)

*  Proxy starterpack requests through ozone and include takedowns

* 🧹 Cleanup

*  Commit snapshots

*  Update snapshot

* lexicon: move starterpacks back to TIDs

* changeset

---------

Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
2024-06-24 21:04:33 -04:00
tsinghuacoder
15cc6ff37c
chore: fix some comments (#2584)
Signed-off-by: tsinghuacoder <tsinghuacoder@icloud.com>
2024-06-20 17:04:47 -07:00
Foysal Ahamed
3ad0519961
Ozone team member manager (#2460)
* 🚧 Proposal for moderator manager lexicons

*  CRUD for moderator management works

*  Add profile view to moderator user list

*  Seed mod users from env var to db

*  Adjust tests

*  Update snapshots

*  Fix type and lexicon token issues

*  Add pagination to listUsers

*  Use sort order in pagination

* 📝 Change error name

*  Update snapshots

*  seed mods in sync to avoid re-order

* 🧹 Remove unnecessary import

* 🐛 Re-run codegen

* 🧹 Some cleanup

* 🧹 Cleanup unnecessary properties on auth-verifier

*  Rename terminologies from moderator to team and member

* 🧹 Cleanup

*  Allow admin tokens to update/add member

*  Delete members in transaction

*  Use db transactions and move profile hydration to the service layer

*  Add test for addMember endpoint

* 🐛 wait on adding admin DID

*  Do not allow updating/deleting service owner and always give service owner admin access

* 🧹 Cleanup

*  Make timestamp columns non null

* 🧹 Cleanup

*  Update mod role definition in getConfig

*  Deletion and update guards

* 🐛 don't prefill service did

* 📝 Add changeset
2024-06-18 18:47:20 -04:00