8 Commits
Author | SHA1 | Message | Date | |
---|---|---|---|---|
rafael
|
1982693e3e
|
Use node:crypto instead of noble/curves (#2936)
* Extract verifySignatureWithKey out of verifyJwt * Accept optional verifySignatureWithKey as param * Impl. verifySignatureWithKey with native crypto * Test key validation * changesets * build * build (fix) * Move verifySig out * Trigger Build * Move test * Remove redundant check --------- Co-authored-by: Devin Ivy <devinivy@gmail.com> |
||
Daniel Holmgren
|
50c0ec176c
|
Service auth method binding (lxm) (#2663)
* add scopes to service auth impl * add error to getServiceAuth * send scoped tokens from pds * clean up privileged access scopes & allow simple service auth tokens for app passwords * integration into ozone * fix up bsky tests * cleanup xrpc-server tests * fix up tests & types * one more test * fix read after write tests * fix mod auth test * convert scopes to be a single method name * add scope check callback for auth verifier * pds changes only * fix feed generation tests * use scope for ozone service profile * dont verify scopes on pds yet * tidy * tidy imports * changeset * add tests * tidy * another changeset * scope -> lxm * tidy * clean up scope references * update nonce size * pr feedback * trim trailing slash * nonce -> jti * fix xrpc-server test * allow service auth on uploadBlob * fix build error * changeset * build, tidy * xrpc-server: update lxm claim check error * appview: temporarily permit labeler service calls to omit lxm claim * xrpc-server: fix test * changeset * fix merged tests --------- Co-authored-by: Devin Ivy <devinivy@gmail.com> |
||
Matthieu Sieben
|
b934b396b1
|
Client SDK rework (#2483)
* feat(api): support creation of oauth based AtpAgents * oauth: misc fixes for confidential clients * fix(xprc): remove ReadableStream.from polyfill * OAuth docs tweaks (#2679) * OAuth: clarification about client_name being shown * OAuth: re-write handle resolution privacy concern * avoid relying on ReadableStream.from in xrpc-server tests * feat(oauth-types): expose "ALLOW_UNSECURE_ORIGINS" constant * feat(handle-resolver): expose "AtprotoIdentityDidMethods" type * fix(oauth-client): ensure that the oauth metadata document contains client_id_metadata_document_supported * fix(oauth-types): prevent unknown query string in loopback client id * fix(identity-resolver): check that handle is in did doc's "alsoKnownAs" * feat(oauth-client:oauth-resolver): allow logging in using either the PDS URL or Entryway URL * fix(oauth-client): return better error in case of invalid "oauth-protected-resource" status code * refactor(did): group atproto specific checks in own * feat(api): relax typing of "appLabelers" and "labelers" AtpClient properties * allow any did as labeller (for tests mainly) * fix(api): allow to override "atproto-proxy" on a per-request basis * remove release candidate versions from changelog * update changeset for api and xrpc packages * Add missing changeset * revert RC versions * Proper wording in OAUTH.md api example * remove "pre" changeset file * xrpc: restore original behavior of setHEader and unsetHeader * docs: add comment for XrpcClient 's constructor arg * feat(api): expose "schemas" publicly * feat(api): allow customizing the whatwg fetch function of the AtpAgent * docs(api): improve migration docs * docs: change reference to BskyAgent to AtpAgent * docs: mention the breaking change regarding setSessionPersistHandler * fix(api): better split AtpClient concerns * fix(xrpc): remove unused import * refactor(api): simplify class hierarchu by removeing AtpClient * fix(api): mock proper method for facets detection * restore ability to restore session asynchronously * feat(api): allow instantiating Agent with same argument as super class * docs(api): properly extend Agent class * style(xrpc): var name * docs(api): remove "async" to header getter --------- Co-authored-by: Devin Ivy <devinivy@gmail.com> Co-authored-by: bnewbold <bnewbold@robocracy.org> Co-authored-by: Hailey <me@haileyok.com> |
||
Daniel Holmgren
|
dc471da267
|
Service auth method binding - PDS (#2668)
* pds changes only * use scope for ozone service profile * dont verify scopes on pds yet * tidy * tidy imports * changeset * add tests * another changeset * scope -> lxm * tidy * update nonce size * pr feedback * trim trailing slash * nonce -> jti * fix xrpc-server test * allow service auth on uploadBlob |
||
Daniel Holmgren
|
f65de89eed
|
Feature: Appview v2 (#1924)
* add buf & connectrpc, codegen client * lint * prettier ignore * fix prettier ignore * tidy & add tests * filler commit * rm filler * server boilerplate * follows impl * posts impl * posts & likes impl * repost impl * profiles & handle null values * list impl * mutes impl * blocks impl * misc * feed gen impl * label impl * notifs impl * feeds impl * threads impl * early sketchwork * wip * stub out thick client * in-progress work on hydrator * tweak * hydrate profile labels, detail lists * feedgen hydration * protobuf tweaks * more protobuf tweaks * wip * snake case * moar snake case * tidy actor hydration * tidy parsing * type fixes, renaming, comments in hydrator * hydrate list items and likes * hydrate notifications * feed hydration * graph & label hydration * more record protobufs * pluralize * tweak pbs * use new methods * Setup dataplane grpc client/mock server (#1921) * add buf & connectrpc, codegen client * lint * prettier ignore * fix prettier ignore * tidy & add tests * add record getter mocks * post view hydration * fix up mock dataplane to match new protos * missed one * wire up dataplane in ctx & dev-env * adding some basic views * feed hydration, add blocks to post hydration * pass over notification hydration * tidy * merge * implement getProfile * hydrate post aggregation and viewer state * fix * fix codegen * get some tests passing! * add takedowns & some like bugfixing * all profile tests passing! * likes test * follow endpoints using data plane * reorg follow block rules * reposts * post views! * implement getList w/ dataplane caveat * adjust dataplane getListMembers to return listitem uris * implement getListMutes and -Blocks w/ dataplane * suggestions * timeline * misc view fixes * view fixes for mutes, self-mute/block * author feed * feed gen routes * tidy * misc block/mute fixes * list feed & actor likes * implement getLists, fix some empty cursors * implement getMutes, empty profile description fix * implement getBlocks, block application fix * implement getSuggestedFollowsByActor, needs some fixes * feed generation * search routes * threads * tidy * fix some snaps * fix getSuggestedFollowsByActor * implement listNotifications * implement getUnreadCount w/ dataplane * implement notifications.updateSeen w/ dataplane * 3rd party blocking tests * blocked profile viewer * add search mocks * refactor getFeed * createPipeline -> createPipelineNew * basic replygating functionality on dataplane w/o filtering violating replies * hack threadgates into dataplane, apply gates * deterministic thread orders in dataplane * misc cleanup around dataplane * upgrade typescript to v5.3 * update typescript linter deps * sync bsky proto, codegen * update dataplane, sync with bsky proto updates * remove indexer, ingester, daemon, moderation services from appview * convert more bsky internals to dataplane, remove custom feedgens, implement mute/unmuting in mock dataplane * remove bsky services. db and indexing logic into mock dataplane. * remove tests not needed by appview v2, misc reorg * add basic in-mem repo subscription to dataplane mock * fix dev-env, bsky tests, bsky build * cull bsky service entrypoint * add bsky service readme * build * tidy * tidy, fix pds proxy tests * fix * fix bsky entrypoint deps * support http2 grpc client * build * fix dataplane bad tls config/default * support multiple dataplane urls, retry when unavailable * build * tidy/fix * move dataplane mock tests into their own dir * cover label hydration through timeline test * bring back labels in appview tests * remove unused db primary/replica/coordinator from bsky dataplane * bsky proto add cids to contracts, buf codegen * sync-up bsky data-plane w/ codegen updates * start using dataplane interaction endpoints * add file * avoid overfetching from dataplane, plumb feed items and cids * pass refs through for post viewer state * switch list feeds to use feed item in dataplane * handle not found err on get-thread dataplane call * support use of search service rather than dataplane methods * mark some appview v2 todos * tidy * still use dataplane on search endpoints when search service is not configured * fix pds test * fix up bsky tests & snaps * tidy migrations * fix appview-v2 docker build * Support label issuer tied to appview v2 (#2033) support label issuer tied to appview * Appview v2: handle empty cursor on list notifications (#2017) handle empty cursor on appview listnotifs * Update appview v2 to use author feed enum (#2047) * update bsky protos with author feed enum, misc feed item changes * support new author feed enums in dataplane * fix build * Appview v2: utilize sorted-at field in bsky protos (#2050) utilize new sorted-at field in bsky protos * remove all dataplane usage of GetLikeCounts, switch to GetInteractionCounts * Appview v2, sync w/ changes to protos (#2071) * sync bsky protos * sync-up bsky implementation w/ proto changes * Appview v2 initial implementation for getPopularFeedGenerators (#2072) add an initial implementation for getPopularFeedGenerators on appview v2 * merge * fixes * fix feed tests * fix bsync mock * format * remove unused config * fix lockfile * another lockfile fix * fix duplicate type * fix dupplicate test * Appview v2 handling clearly bad cursors (#2092) * make mock dataplane cursors different from v1 cursors * fail open on clearly bad appview cursors * fix pds appview proxy snaps * Appview v2 no notifs seen behavior (#2096) * alter behavior for presenting notifications w/ no last-seen time * fix pds proxy tests * Appview v2 dataplane retries based on client host (#2098) choose dataplane client for retries based on host when possible/relevant * don't apply negated labels * display suspensions on actor profile in appview v2 * Appview v2 use dataplane for identity lookups (#2095) * update bsky proto w/ identity methods * setup identity endpoints on mock dataplane * move from idresolver to dataplane for identity lookups on appview * tidy * Appview v2: apply safe takedown refs to records, actors (#2107) apply safe takedown refs to records, actors * Fix timing on appview v2 repo rev header (#2113) fix timing on appview repo rev * fix post thread responses * Appview v2 don't apply 3p self blocks (#2112) do not apply 3p self-blocks * Appview v2 search for feed generators (#2118) * add protos for feedgen search * support feed search on getPopularFeedGenerators * Appview v2 config tidy (#2117) * remove mod and triage roles from appview * rename cdn and search config * remove custom feed harness from appview v2 * Appview v2: don't apply missing modlists (#2122) * dont apply missing mod lists * update mock dataplane * Update packages/bsky/src/hydration/hydrator.ts Co-authored-by: devin ivy <devinivy@gmail.com> * refactor & document a bit better * fix up other routes --------- Co-authored-by: devin ivy <devinivy@gmail.com> * Appview v2 enforce post thread root boundary (#2120) * enforce post thread root boundary * test thread root boundary * Appview v2 fix admin environment variable (#2137) fix admin env in appview v2 * Remove re-pagination from getSuggestions (#2145) * remove re-pagination from getSuggestions * fix test * Adjust wording for account suspension (#2153) adjust wording for account suspension * Appview v2: fix not-found and blocked uris in threads (#2201) * fix uris of not-found and blocked posts in threads * update snaps * ✨ Show author feed of takendown author to admins only (#2197) * fold in cid, auth, tracing, node version changes * remove dead config from bsky service entrypoint * build * remove ozone test codepaths for appview v2 * tidy, docs fix --------- Co-authored-by: Devin Ivy <devinivy@gmail.com> Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz> |
||
Daniel Holmgren
|
de2dbc2903
|
Split out moderation backend (#1970)
* mv appview * copy * finalize copy * package names * big WIP * first pass at mod servce * some tidy * tidy & fix compiler errors * rename to ozone, db migrations, add to dev-env & pds cfg * getRecord & getRepo mostly working * fix open handle * get record tests all working * moderation events working * statuses working * tidy test suite * search repos * server & db tests * moderation tests * wip daemon + push events * pds fanout working * fix db test * fanning takedowns out to appview * rm try/catch * bsky moderation test * introduce mod subject wrappers * more tidy * refactor event reversal * tidy some db stuff * tidy * rename service to mod-service * fix test * tidy config * refactor auth in bsky * wip patching up auto-mod * add label ingester in appview * fix a couple build issues * fix some timing bugs * tidy polling logic * fix up tests * fix some pds tests * eslint ignore * fix ozone tests * move seeds to dev-env * move images around * fix db schemas * use service auth admin reqs * fix remaining tests * auth tests bsky * another test * random tidy * fix up search * clean up bsky mod service * more tidy * default attempts to 0 * tidy old test * random tidy * tidy package.json * tidy logger * takedownId -> takedownRef * misc pr feedback * split daemon out from ozone application * fix blob takedown mgiration * refactor ozone config * do push event fanout on write instead of on read * make suspend error work again * add attempts check & add supporting index * fix takedown test ref * get tests working * rm old test * fix timing bug in event pusher tests * attempt another fix for timing bug * await req * service files * remove labelerDid cfg * update snaps for labeler did + some cfg changes * fix more snaps * pnpm i * build ozone images * build * make label provider optional * fix build issues * fix build * fix build * build pds * build on ghcr * fix syntax in entry * another fix * use correct import * export logger * remove event reverser * adjust push event fanout * push out multiple * remove builds |
||
Daniel Holmgren
|
f9fd3e68ca
|
Feature branch: PDS v2 (#1789)
* cleanup repeat process all * wip * skip actor search test * skip actor search test * tweak processAll * decrease wait to 1 sec * repo_blob -> record_blob * simplify backlink linkTo * return repo_root to one row * sequence before updating repo_root * invite code forUser -> forAccount * ipld_block -> repo_block * use lru-cache fetchMethod * move did_cache to own db * better error handling on did cache * drop did_handle * fix sequencer wait time * debug * debug * more debug * check something * fix bday paradox * fix bday paradox * tidy up pds service auth * rm skipped test * retry http * tidy * improve fanout error handling * fix test * return signing key in did-web * more tests * tidy serivce auth checks * user_account -> account * remove inviteNote * keypair per repo * use an lru cache for keypairs as well * clean up repo * wip * wrap up accoutn manager * tidy * tidy * fix tests * fix disabled codes * fix appview tests * add note * set pragmas * tidy account manager getDb * rename pref transactor * user pref -> account pref * handle blob imports * tidy imports * add reserveSigningKey * wip transferAccount * clean up transferAccount * tests * tidy * tidy * configure entryway url on pds * handle entryway in pds admin endpoints * make importRepo temp * fix imports * make email optional on pds when using entryway * handle diffs * handle pds entryway usage for server, identity, admin endpoints * pds support for credentials from entryway * setup pds tests w/ entryway service * tidy * tidy * update entryway version * wip * test handle updates w/ entryway * split account table into two * tidy * tweak scripts * tidy tests * tidy * better config for actorstore & dbs * clean up cfg more * reorg actorstore fs layout * handle erros on actor db create * pr tidy & fix accoutn deletion test * pr feedback * fix bad merge * unskip test * fix subscribe repos tests * tidy repo root tables * tidy * fix tests * tidy delete tokens * tidy account getters * tidy * bulk deletesg * increase chunk size * handle racing refreshes * wip * fix auth test * invert import flow * clean up actor store on create account failure * tweak sequencer * prevent invite code races on createAccount * rm note * add back in race protection on getAccountInviteCodes * start feature branch * deleted app migration table * patch up new auth test * rm note * g * create accoutn delegated from entryway * tidy * fix test * change plcOp type to unknown * small fixes * sync up w entryway branch * Use proper error when authed account is not found (#1799) provide proper error when account not found in access-takedown check * build branch * build on ghcr * tweak service file * tweak service file * change where we save reserved keys * no tmp dir in blobstore either * fix blobstore temp location again * handle repeat record_blobs * create account before submitting plc op & undo if fail * small tweak * limit the number of local records * push out empty commit on transfer * fix issue with record_blob * add push blob endpoint * Set and validate token audiences on pds v2 (#1793) set and validate token audience on pds v2 * merge * include entryway did on tests * build branch * fix cache issue * xrpc server blob limit * put correct bytes * add auth to routes * handle quarantining/unquarantining a blob that does not exist * tidy * fix transfer tests * fix email request routes for entryway * PDS v2 entryway account deletion (#1819) * add admin lexicon for account deletion * implement admin account deletion endpoint * fix entryway proxying on account email checks * proxy to entryway for acct deletion * read-after-write sanity check * tweak * wip * finish refactor * fix test schema * application retry logic for busy * pr feedback * rm lru-cache * fix test pg schema * fix transfer test * Sqlite instrumentation for pds v2 (#1838) * sqlite instrumentation * build * remove build * dont reimport blobs * send ticks during import * close on error * catch handle validation error * add log * fix test * return emailConfirmedAt on getAccountInfo * Upgrade sharp on pds v2 (#1863) upgrade sharp to 0.32.6 * read all bytes before parsing car * Async car reader (#1867) * asynchronously read in car * dont buffer car * tweak * Gracefully handle indexing of invalid records (#1853) * gracefully handle indexing of invalid records * fix repo tests * Fix role auth for access-or-role verifier, getBlob check on actor takedowns (#1869) fix role auth for access-or-role verifier, fix getBlob actor takedown check * better cleanup of actor-stores * add ability to not ensure leaves * tidy * allow did:web transfer * Migration utility for actor-store (#1873) beginnings of helper for migrating all actors Co-authored-by: Devin Ivy <devinivy@gmail.com> * base case for findBlobRefs * App-level retries for sqlite on pds (#1871) * revamp retry helper to be more flexible re: backoff strategies * sqlite timeout helper * ensure sqlite wal on db creation/migration rather than every open * layer retries for sqlite on writes outside transactions on pds * tidy * fix up lockfile * tidy * fix lex codegen * fix timing bug in threadgate test * No-op update handling (#1916) do no produce commits on no-op updates * Retry on all SQLITE_BUSY error codes (#1917) retry on all sqlite_busy error codes * Pds v2 ensure sqlite ready (#1918) ensure sqlite is ready before making queries * try something * tidy * dont build branch --------- Co-authored-by: Devin Ivy <devinivy@gmail.com> |
||
devin ivy
|
8637c367fe
|
Respect updated service auth keys (#1765)
* bust key cache when verifying service auth * unit tests for xrpc auth * fix * support option for verifying non-low-s signatures * fix verifyJwt tests |