8 Commits

Author SHA1 Message Date
devin ivy
71785d31f7
Ozone: pass through unknown account details (#2794)
* ozone: pass through account info details that may not be known yet

* tidy, build
2024-09-05 21:22:48 -04:00
Daniel Holmgren
50c0ec176c
Service auth method binding (lxm) (#2663)
* add scopes to service auth impl

* add error to getServiceAuth

* send scoped tokens from pds

* clean up privileged access scopes & allow simple service auth tokens for app passwords

* integration into ozone

* fix up bsky tests

* cleanup xrpc-server tests

* fix up tests & types

* one more test

* fix read after write tests

* fix mod auth test

* convert scopes to be a single method name

* add scope check callback for auth verifier

* pds changes only

* fix feed generation tests

* use scope for ozone service profile

* dont verify scopes on pds yet

* tidy

* tidy imports

* changeset

* add tests

* tidy

* another changeset

* scope -> lxm

* tidy

* clean up scope references

* update nonce size

* pr feedback

* trim trailing slash

* nonce -> jti

* fix xrpc-server test

* allow service auth on uploadBlob

* fix build error

* changeset

* build, tidy

* xrpc-server: update lxm claim check error

* appview: temporarily permit labeler service calls to omit lxm claim

* xrpc-server: fix test

* changeset

* fix merged tests

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2024-08-18 15:46:07 -04:00
Daniel Holmgren
2ea27e340a
Ozone lexicons (#2288)
* lexicons

* clean up old admin schemas

* codegen

* renaming all routes

* fix more

* fix snaps

* fix test

* format

* hook up ozone routes on pds

* publish next version

* build branch

* dont build branch
2024-03-12 18:42:11 -05:00
Foysal Ahamed
2802880a97
Add modEventDivertBlobs event to send blobs to abyss (#2238)
* 🚧 Working through an nullable review state

*  Update snapshots on some tests

*  Update snapshots on some tests

*  Add test for reviewOptional status mutation

*  Add divertBlobs event to send blobs to abyss

* ♻️ Rename reviewOptional -> reviewNone

* ♻️ Rename modEventDivertBlobs -> modEventDivert

* 🐛 Rename event type checker

*  Use pds resolver to get blob straight from pds

*  Use FOR UPDATE to respect db transactions

* ♻️ Refactor to use event_pusher table instead of new table

*  Bring back missing lines in pnpm-lock

* 🔨 Rebuild?

* 🚨 Formatting

* ♻️ Refactor to divert blob sync

* 🧹 Cleanup

*  Use modClient seed client in blob-divert test

* update divert blob config to use basic admin auth

* fix

* build

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2024-03-12 10:40:32 -04:00
Daniel Holmgren
71f9cc9d4f
Clean up role-based auth (#2255)
* tidy bsky auth

* hook up new auth verifier

* update auth throughout ozone

* handle mod signing keys

* add client proxy heads to pds

* hook up rest of routes

* simplify pipethrough & add some SSRF protection

* tests

* fix bad var

* remove basic auth in ozone

* wip

* fix key parsing in pds

* fix up all ozone tests

* fix admin auth test

* rename test

* fix ozone test

* clean up tokens in pds

* fix up pds tests

* fix up ozone tests

* add pipethrough to write routes

* reenable proxied admin test

* add moderator accounts to ozone in dev-env

* update did doc id values

* null creds string -> `none`

* fix fetchLabels auth check

*  Add a couple more proxied requests that we use in ozone ui

* Add runit to the services/bsky Dockerfile (#2254)

add runit to the services/bsky Dockerfile

* Improve tag detection (#2260)

* Allow tags to lead with and contain only numbers

* Break tags on other whitespace characters

* Export regexes from rich text detection

* Add test

* Add test

* Disallow number-only tags

* Avoid combining enclosing screen chars

* Allow full-width number sign

* Clarify tests

* Fix punctuation edge case

* Reorder

* Simplify, add another test

* Another test, comment

* Version packages (#2261)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* 🐛 Increment attempt count after each attempt to push ozone event (#2239)

* Ozone delegates email sending to actor's pds (#2272)

* ozone delegates email sending to user's pds

* lexicon: add content field to mod email event

* test email sending via mod event

* add dev dep for nodemailer in ozone

* fix auth verifier method

* build branch

* build branch

* fix url check

* better error handling for get account infos

* fix labeler service id

* fix iss on auth headers

* fix dev-env ozone did

* fix tests & another jwt issuer

* fix proxy auth

* ozone: fix ip check

* fix aud check on pds mod service auth

* tidy

* Update packages/pds/tests/proxied/admin.test.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* fix pipethrough of headers

* fix moderation status tests

* fix auth on ozone routes

* update iss on daemon

---------

Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
Co-authored-by: Jake Gold <52801504+Jacob2161@users.noreply.github.com>
Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: devin ivy <devinivy@gmail.com>
2024-03-06 19:53:52 -06:00
Daniel Holmgren
2267f1ec48
Add client-set proxy headers to PDS (#2251)
* tidy bsky auth

* hook up new auth verifier

* update auth throughout ozone

* handle mod signing keys

* add client proxy heads to pds

* hook up rest of routes

* simplify pipethrough & add some SSRF protection

* tests

* fix bad var

* fix key parsing in pds

* fix admin auth test

* rename test

* add pipethrough to write routes

* update did doc id values

* null creds string -> `none`

* fix fetchLabels auth check

*  Add a couple more proxied requests that we use in ozone ui

* Add runit to the services/bsky Dockerfile (#2254)

add runit to the services/bsky Dockerfile

* Improve tag detection (#2260)

* Allow tags to lead with and contain only numbers

* Break tags on other whitespace characters

* Export regexes from rich text detection

* Add test

* Add test

* Disallow number-only tags

* Avoid combining enclosing screen chars

* Allow full-width number sign

* Clarify tests

* Fix punctuation edge case

* Reorder

* Simplify, add another test

* Another test, comment

* Version packages (#2261)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* 🐛 Increment attempt count after each attempt to push ozone event (#2239)

* Ozone delegates email sending to actor's pds (#2272)

* ozone delegates email sending to user's pds

* lexicon: add content field to mod email event

* test email sending via mod event

* fix auth verifier method

* build branch

* fix url check

* better error handling for get account infos

* fix labeler service id

* fix iss on auth headers

* fix dev-env ozone did

* fix tests & another jwt issuer

* ozone: fix ip check

* fix aud check on pds mod service auth

* tidy

* fix pipethrough of headers

---------

Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
Co-authored-by: Jake Gold <52801504+Jacob2161@users.noreply.github.com>
Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: devin ivy <devinivy@gmail.com>
2024-03-06 19:38:21 -06:00
Daniel Holmgren
81370d7618
Ozone cdn invalidation (#2087)
* hooking up invalidator to ozone

* test + fix

* wire up service entry

* add cid to invalidation url

* add aws to ozone service package.json

* build branch

* fix build

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2024-02-29 15:28:26 -05:00
Daniel Holmgren
de2dbc2903
Split out moderation backend (#1970)
* mv appview

* copy

* finalize copy

* package names

* big WIP

* first pass at mod servce

* some tidy

* tidy & fix compiler errors

* rename to ozone, db migrations, add to dev-env & pds cfg

* getRecord & getRepo mostly working

* fix open handle

* get record tests all working

* moderation events working

* statuses working

* tidy test suite

* search repos

* server & db tests

* moderation tests

* wip daemon + push events

* pds fanout working

* fix db test

* fanning takedowns out to appview

* rm try/catch

* bsky moderation test

* introduce mod subject wrappers

* more tidy

* refactor event reversal

* tidy some db stuff

* tidy

* rename service to mod-service

* fix test

* tidy config

* refactor auth in bsky

* wip patching up auto-mod

* add label ingester in appview

* fix a couple build issues

* fix some timing bugs

* tidy polling logic

* fix up tests

* fix some pds tests

* eslint ignore

* fix ozone tests

* move seeds to dev-env

* move images around

* fix db schemas

* use service auth admin reqs

* fix remaining tests

* auth tests bsky

* another test

* random tidy

* fix up search

* clean up bsky mod service

* more tidy

* default attempts to 0

* tidy old test

* random tidy

* tidy package.json

* tidy logger

* takedownId -> takedownRef

* misc pr feedback

* split daemon out from ozone application

* fix blob takedown mgiration

* refactor ozone config

* do push event fanout on write instead of on read

* make suspend error work again

* add attempts check & add supporting index

* fix takedown test ref

* get tests working

* rm old test

* fix timing bug in event pusher tests

* attempt another fix for timing bug

* await req

* service files

* remove labelerDid cfg

* update snaps for labeler did + some cfg changes

* fix more snaps

* pnpm i

* build ozone images

* build

* make label provider optional

* fix build issues

* fix build

* fix build

* build pds

* build on ghcr

* fix syntax in entry

* another fix

* use correct import

* export logger

* remove event reverser

* adjust push event fanout

* push out multiple

* remove builds
2024-01-05 17:06:54 -06:00