11 Commits

Author SHA1 Message Date
Foysal Ahamed
c559e7e3de
Add subject tags with report type (#3260)
*  Add subject tags with report type

*  Update snapshots

*  Update snpshot for pds admin

*  Update ozone snapshot

*  Update ozone snapshot

* 🧹 Cleanup
2024-12-18 22:57:27 +00:00
Foysal Ahamed
d6f33b4742
Add events for account and record update/delete/deactivation (#2661)
*  Add events for account and record update/delete/deactivation

*  Add handle change event

*  Reduce account events to 2 types and record events to 1

*  Store metadata from account, identity and record events

*  Add created event for record

*  Add ndd the new events to allowed types in emitEvent

*  Use string value for record op and add tombstone flag to identity event

*  Add active flag on account events

*  Change accountStatus -> status to match with firehose event

*  Make active flag required

* 🚨 fix prettier style issue

*  Track record/account delete and update data in subject status (#2804)

*  Store deleted/updated event data in subject_status

* 🐛 Fix query for recordDeletedAt and recordUpdatedAt

*  Add tombstoned status

*  Move from record to hosting term

*  Add tests for hosting params

*  Update lexicons for hostingStatuses

*  Update snapshots

*  Update snapshots

*  Update snapshots

*  Adjust hosting statuses

* 📝 Add changeset
2024-11-07 22:51:44 +01:00
devin ivy
af7d3912a3
Starter packs (#2553)
* lexicon: initial starter pack record lexicons

* lexicon: first pass on views for starter packs

* lexicon: add starter pack query endpoints, misc fixes

* lexicon: add starter pack related fields to profile views

* lexicon: codegen

* api prerelease

* bsky: protos for starter packs

* appview: implement starter pack endpoints in dataplane, protos update

* appview: starter pack hydration and view, mock dataplane indexing

* appview: wire-up starter pack graph endpoints

* appview: test basic starter packs functionality, update dev-env

* fix snaps for profile createdAt

* appview: fix list item count for starter packs in dataplane

* fix ozone snaps, tidy

* appview: reorg list aggs for starter packs

* appview: starter pack lexicon and proto tweaks, move around aggregations

* appview: fix views and snaps

* api prerelease

* fix snaps

* appview: misc proto updates for starter packs, snaps

* appview: add starter pack counts to profile detail

* build

* api prerelease

* pds: starter pack slur detection

* lexicon: starter pack notifications and batch fetch

* appview: starter pack notification and batch fetch

* appview: add starterpack-joined notif to dataplane and test

* api prerelease

* ozone: fix snaps

* pds build

*  Proxy starterpack requests through ozone and include takedowns (#2594)

*  Proxy starterpack requests through ozone and include takedowns

* 🧹 Cleanup

*  Commit snapshots

*  Update snapshot

* lexicon: move starterpacks back to TIDs

* changeset

---------

Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
2024-06-24 21:04:33 -04:00
Daniel Holmgren
069d1432dc
Remove false neg on the labels that ozone emits (#2310)
* remove `neg` on the labels that ozone emits

* no negs in appview either

* no negs on self-labels

* no neg on self labels in ozone either

* fix query labels test
2024-03-13 17:33:50 -05:00
Daniel Holmgren
16ac2ef0b3
Add specced takedown labels (#2309)
* add specced takedown labels

* fix up tests

* fix test

* only reverse takedown labels
2024-03-13 17:33:43 -05:00
Daniel Holmgren
2ea27e340a
Ozone lexicons (#2288)
* lexicons

* clean up old admin schemas

* codegen

* renaming all routes

* fix more

* fix snaps

* fix test

* format

* hook up ozone routes on pds

* publish next version

* build branch

* dont build branch
2024-03-12 18:42:11 -05:00
Daniel Holmgren
cc11adda87
Ozone: Add sigs to all labels (#2236)
* sketching out label sequencer

* refactor sequencer

* sequencer tests

* tests

* add query labels endpoint & tests

* add pagination

* fix label formatting on temp

* tidy

* format labels

* make use listen/notify for sequencer

* ensure sig on all outgoing labels from ozone

* fixing up tests

* fix sequencer tests

* fix hanging server test

* add log on failure to update label

* update description for sig

* use bytes for label sigs

* fix tests

* add ver to labels

* tidy up background queue

* store signing keys as ids

* fix sequencer teest
2024-03-08 19:41:31 -06:00
Daniel Holmgren
71f9cc9d4f
Clean up role-based auth (#2255)
* tidy bsky auth

* hook up new auth verifier

* update auth throughout ozone

* handle mod signing keys

* add client proxy heads to pds

* hook up rest of routes

* simplify pipethrough & add some SSRF protection

* tests

* fix bad var

* remove basic auth in ozone

* wip

* fix key parsing in pds

* fix up all ozone tests

* fix admin auth test

* rename test

* fix ozone test

* clean up tokens in pds

* fix up pds tests

* fix up ozone tests

* add pipethrough to write routes

* reenable proxied admin test

* add moderator accounts to ozone in dev-env

* update did doc id values

* null creds string -> `none`

* fix fetchLabels auth check

*  Add a couple more proxied requests that we use in ozone ui

* Add runit to the services/bsky Dockerfile (#2254)

add runit to the services/bsky Dockerfile

* Improve tag detection (#2260)

* Allow tags to lead with and contain only numbers

* Break tags on other whitespace characters

* Export regexes from rich text detection

* Add test

* Add test

* Disallow number-only tags

* Avoid combining enclosing screen chars

* Allow full-width number sign

* Clarify tests

* Fix punctuation edge case

* Reorder

* Simplify, add another test

* Another test, comment

* Version packages (#2261)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* 🐛 Increment attempt count after each attempt to push ozone event (#2239)

* Ozone delegates email sending to actor's pds (#2272)

* ozone delegates email sending to user's pds

* lexicon: add content field to mod email event

* test email sending via mod event

* add dev dep for nodemailer in ozone

* fix auth verifier method

* build branch

* build branch

* fix url check

* better error handling for get account infos

* fix labeler service id

* fix iss on auth headers

* fix dev-env ozone did

* fix tests & another jwt issuer

* fix proxy auth

* ozone: fix ip check

* fix aud check on pds mod service auth

* tidy

* Update packages/pds/tests/proxied/admin.test.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* fix pipethrough of headers

* fix moderation status tests

* fix auth on ozone routes

* update iss on daemon

---------

Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
Co-authored-by: Jake Gold <52801504+Jacob2161@users.noreply.github.com>
Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: devin ivy <devinivy@gmail.com>
2024-03-06 19:53:52 -06:00
Foysal Ahamed
d81573232b
Allow filtering moderation queue by language (#2161)
*  Store languages on subjects from record data

*  Fetch lang from author feed for repo subject

*  Regenerate lex

*  Add lang to profile subject

*  Move lang to flag

*  Add comment to flag event

*  Update pds test snapshot

* 🐛 Fix broken import

*  Rename flag to tag

*  Update snapshot for event by id

*  Bring back skipped test

*  Move tags to dedicated columns

*  Update test snapshots, use logger

*  Change lang:unknown to lang:und

*  Update test snapshots

* 🧹 Cleanup

*  Add filter params for tags on queryModerationEvents endpoint

*  Add tags property to subject status model

*  Update test code

*  Add filter to get subjects that do not have certain tags
2024-02-19 10:35:03 -05:00
Daniel Holmgren
c048962632
Ozone !takedown labels (#2069)
* add purge labels on takedown

* wire up some cfg

* tests

* fix up tests

* purge -> takedown

* Temporary unspecced labels relating takedowns in ozone (#2101)

temporarily move to unspecced labels relating takedowns

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2024-01-27 17:29:36 -05:00
Daniel Holmgren
de2dbc2903
Split out moderation backend (#1970)
* mv appview

* copy

* finalize copy

* package names

* big WIP

* first pass at mod servce

* some tidy

* tidy & fix compiler errors

* rename to ozone, db migrations, add to dev-env & pds cfg

* getRecord & getRepo mostly working

* fix open handle

* get record tests all working

* moderation events working

* statuses working

* tidy test suite

* search repos

* server & db tests

* moderation tests

* wip daemon + push events

* pds fanout working

* fix db test

* fanning takedowns out to appview

* rm try/catch

* bsky moderation test

* introduce mod subject wrappers

* more tidy

* refactor event reversal

* tidy some db stuff

* tidy

* rename service to mod-service

* fix test

* tidy config

* refactor auth in bsky

* wip patching up auto-mod

* add label ingester in appview

* fix a couple build issues

* fix some timing bugs

* tidy polling logic

* fix up tests

* fix some pds tests

* eslint ignore

* fix ozone tests

* move seeds to dev-env

* move images around

* fix db schemas

* use service auth admin reqs

* fix remaining tests

* auth tests bsky

* another test

* random tidy

* fix up search

* clean up bsky mod service

* more tidy

* default attempts to 0

* tidy old test

* random tidy

* tidy package.json

* tidy logger

* takedownId -> takedownRef

* misc pr feedback

* split daemon out from ozone application

* fix blob takedown mgiration

* refactor ozone config

* do push event fanout on write instead of on read

* make suspend error work again

* add attempts check & add supporting index

* fix takedown test ref

* get tests working

* rm old test

* fix timing bug in event pusher tests

* attempt another fix for timing bug

* await req

* service files

* remove labelerDid cfg

* update snaps for labeler did + some cfg changes

* fix more snaps

* pnpm i

* build ozone images

* build

* make label provider optional

* fix build issues

* fix build

* fix build

* build pds

* build on ghcr

* fix syntax in entry

* another fix

* use correct import

* export logger

* remove event reverser

* adjust push event fanout

* push out multiple

* remove builds
2024-01-05 17:06:54 -06:00