modify-pds
25 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Foysal Ahamed
|
6d308b857b |
✨ Allow appeals on takendown account (#3251)
* ✨ Allow appeals on takendown account * ✅ Update snapshot * ✅ Remove duplicate test * ✨ Respond with takendown token from createSession for takendown accounts * 🧹 cleanup appeal account action stuff * 📝 Add description to new field * ♻️ Refactor authscope formatter and add test for create record with takendown token * ✅ Update snapshot * add createReport route * changeset --------- Co-authored-by: dholms <dtholmgren@gmail.com> |
||
Matthieu Sieben
|
ebb318325b |
Improved control over JWT's typ claim (#2743)
* Add "jwtAlg" option to verifySignature() function * Verify service JWT header values. Add iat claim to service JWT * Allow missing 'typ' claim in service auth jwt * Add, and verify, a "typ" header to access and refresh tokens * tidy * Properly identify JWT typ missmatch * tidy * exclude known invalid "typ" from service auth headers * tidy * tidy changeset --------- Co-authored-by: devin ivy <devinivy@gmail.com> |
||
|
Matthieu Sieben
|
b934b396b1 |
Client SDK rework (#2483)
* feat(api): support creation of oauth based AtpAgents * oauth: misc fixes for confidential clients * fix(xprc): remove ReadableStream.from polyfill * OAuth docs tweaks (#2679) * OAuth: clarification about client_name being shown * OAuth: re-write handle resolution privacy concern * avoid relying on ReadableStream.from in xrpc-server tests * feat(oauth-types): expose "ALLOW_UNSECURE_ORIGINS" constant * feat(handle-resolver): expose "AtprotoIdentityDidMethods" type * fix(oauth-client): ensure that the oauth metadata document contains client_id_metadata_document_supported * fix(oauth-types): prevent unknown query string in loopback client id * fix(identity-resolver): check that handle is in did doc's "alsoKnownAs" * feat(oauth-client:oauth-resolver): allow logging in using either the PDS URL or Entryway URL * fix(oauth-client): return better error in case of invalid "oauth-protected-resource" status code * refactor(did): group atproto specific checks in own * feat(api): relax typing of "appLabelers" and "labelers" AtpClient properties * allow any did as labeller (for tests mainly) * fix(api): allow to override "atproto-proxy" on a per-request basis * remove release candidate versions from changelog * update changeset for api and xrpc packages * Add missing changeset * revert RC versions * Proper wording in OAUTH.md api example * remove "pre" changeset file * xrpc: restore original behavior of setHEader and unsetHeader * docs: add comment for XrpcClient 's constructor arg * feat(api): expose "schemas" publicly * feat(api): allow customizing the whatwg fetch function of the AtpAgent * docs(api): improve migration docs * docs: change reference to BskyAgent to AtpAgent * docs: mention the breaking change regarding setSessionPersistHandler * fix(api): better split AtpClient concerns * fix(xrpc): remove unused import * refactor(api): simplify class hierarchu by removeing AtpClient * fix(api): mock proper method for facets detection * restore ability to restore session asynchronously * feat(api): allow instantiating Agent with same argument as super class * docs(api): properly extend Agent class * style(xrpc): var name * docs(api): remove "async" to header getter --------- Co-authored-by: Devin Ivy <devinivy@gmail.com> Co-authored-by: bnewbold <bnewbold@robocracy.org> Co-authored-by: Hailey <me@haileyok.com> |
||
Daniel Holmgren
|
255d5ea1f0 |
Account deactivation (#2531)
* subscribe repos lex: new #account event type * lex: getAccountStatus endpoint * lex: add account status errors to sync methods * tweak type of token union * fix getAccountStatus parameter name * codegen * Account -> Repo * re-codegen * update errors in sync methods * add getRepoStatus route * add account events to sequencer * emit account evts * fix test + small bugfixin * handle evt on bsky side * codegen * loggable message * schema tweaks * build errors & tidy * tidy account deactivation tests * more subscribe repos tests * identity evt tests + tidy * return optional did doc on identity evts * update identity evt * update impl for identity evt handles * add description to handle field * status on listRepos & notate deprecated firehose events * return status on listRepos * pr feedback * tests * protos * add status to session schemas * codegen * revamp auth-verifier * thread through agent * fix tests * fix account deletion test * fix more tests * fix type error in repo.test * bsky index account evts * apply in appview * add to admin route * add deactivatedAt to ozone views * pds tests * appview tests * ozone test * fix bsky test * fix agent test * fix indexing test * tweak session schemas * update session routes & clean up status handling in pds & agent * changeset * patch pds test * rm active from knownValues --------- Co-authored-by: bryan newbold <bnewbold@robocracy.org> |
||
|
Matthieu Sieben
|
f689bd51a2 |
Build system rework (#2169)
* refactor(crypto): remove circular dependency * refactor(crypto): expose compress/decompress as part of the DidKeyPlugin interface * fix(crypto): remove import from private file * refactor: isolate tsconfig * fix: remove unused bench file * chore(repo): remove unused deps * fix(ozone): properly list dependencies * fix(services): do lint js files * fix(services/pds): remove unused deps * chore(pds): remove bench * chore(dev-env): remove unused deps * chore(api): remove bench * remove unused babel.config.js files * fix: remove .ts extension from import * fix(pds): remove imports of src files * fix(tsconfig): properly list all projects * fix(dev-env): remove imports of src files * fix(bsky): remove direct import to crypto src * fix(api): remove imports to api internals * chore(build): prevent bundling of built output * chore(dev): add "dev" script to build in watch mode * chore(deps): move ts-node dependency where it is actually used * fix(deps): add dev-env as project dependency * fix(xrpc-server): properly type kexicon * fix(bsky): improve typings * fix(pds): fully type formatRecordEmbedInternal return value * fix(repo): remove imports from @ipld/car/api * feat(dev-env): re-export BskyIngester * fix: properly lint & type jest config & test files * fix(ci): test after build * fix(types): use NodeJS.Timeout instead of NodeJS.Timer * fix(bsky): make types exportable * fix(ozone): make types exportable * fix(xrpc-server): make types exportable * fix(xprc-server): make code compliant with "node" types * fix(xrpc-server): avoid accessing properties of unknown * chore(deps): update @types/node * feat(tsconfig): narrow down available types depending on the package's target environment * fix(pds): remove unused prop * fix(bsync): Database's migrator not always initialized * fix(dev-env): remove unreachable code * fix(xrpc-server): remove unused import * fix(xrpc-server): mark header property as abstract * fix(pds): initialize LeakyTxPlugin's txOver property * fix(bsky): initialize LeakyTxPlugin's txOver property * fix(bsky): remove unused migrator from DatabaseCoordinator * fix(bsky): Properly initialize LabelService's cache property * fix(ozone): Database's migrator not initialized * fix(ozone): initialize LeakyTxPlugin's txOver property * fix(crypto): ignore unused variable error * feat(tsconfig): use stricter rules * feat(tsconfig): enable useDefineForClassFields * feat(xrpc-server): add support for brotli incoming payload * fix(xrpc-server): properly parse & process content-encoding * fix(common:stream): always call cb in _transform * tidy/fix tests and service entrypoints * Revert "fix(xrpc-server): properly parse & process content-encoding" This reverts commit 2b1c66e153820d3e128fc839fcc1834d52a66686. * Revert "feat(xrpc-server): add support for brotli incoming payload" This reverts commit e710c21e6118214ddf215b0515e68cb87299a952. * remove special node env for tests (defaults to jest val of "test") * kill mute sync handler on disconnect * work around connect-es bug w/ request aborts * style(crypto): rename imports from uint8arrays * fix update package-lock * fix lint * force hbs files to be bundled as cjs * fix: use concurrently instead of npm-run-all npm-run-all seems not to be maintained anymore. Additionally, concurrently better forwards signals to child processes. * remove concurrently alltogether * ignore sqlite files in services/pds * fix verify * fix verify * tidy, fix verify * fix blob diversion test * build rework changeset --------- Co-authored-by: Devin Ivy <devinivy@gmail.com> |
||
|
Daniel Holmgren
|
f9fd3e68ca |
Feature branch: PDS v2 (#1789)
* cleanup repeat process all * wip * skip actor search test * skip actor search test * tweak processAll * decrease wait to 1 sec * repo_blob -> record_blob * simplify backlink linkTo * return repo_root to one row * sequence before updating repo_root * invite code forUser -> forAccount * ipld_block -> repo_block * use lru-cache fetchMethod * move did_cache to own db * better error handling on did cache * drop did_handle * fix sequencer wait time * debug * debug * more debug * check something * fix bday paradox * fix bday paradox * tidy up pds service auth * rm skipped test * retry http * tidy * improve fanout error handling * fix test * return signing key in did-web * more tests * tidy serivce auth checks * user_account -> account * remove inviteNote * keypair per repo * use an lru cache for keypairs as well * clean up repo * wip * wrap up accoutn manager * tidy * tidy * fix tests * fix disabled codes * fix appview tests * add note * set pragmas * tidy account manager getDb * rename pref transactor * user pref -> account pref * handle blob imports * tidy imports * add reserveSigningKey * wip transferAccount * clean up transferAccount * tests * tidy * tidy * configure entryway url on pds * handle entryway in pds admin endpoints * make importRepo temp * fix imports * make email optional on pds when using entryway * handle diffs * handle pds entryway usage for server, identity, admin endpoints * pds support for credentials from entryway * setup pds tests w/ entryway service * tidy * tidy * update entryway version * wip * test handle updates w/ entryway * split account table into two * tidy * tweak scripts * tidy tests * tidy * better config for actorstore & dbs * clean up cfg more * reorg actorstore fs layout * handle erros on actor db create * pr tidy & fix accoutn deletion test * pr feedback * fix bad merge * unskip test * fix subscribe repos tests * tidy repo root tables * tidy * fix tests * tidy delete tokens * tidy account getters * tidy * bulk deletesg * increase chunk size * handle racing refreshes * wip * fix auth test * invert import flow * clean up actor store on create account failure * tweak sequencer * prevent invite code races on createAccount * rm note * add back in race protection on getAccountInviteCodes * start feature branch * deleted app migration table * patch up new auth test * rm note * g * create accoutn delegated from entryway * tidy * fix test * change plcOp type to unknown * small fixes * sync up w entryway branch * Use proper error when authed account is not found (#1799) provide proper error when account not found in access-takedown check * build branch * build on ghcr * tweak service file * tweak service file * change where we save reserved keys * no tmp dir in blobstore either * fix blobstore temp location again * handle repeat record_blobs * create account before submitting plc op & undo if fail * small tweak * limit the number of local records * push out empty commit on transfer * fix issue with record_blob * add push blob endpoint * Set and validate token audiences on pds v2 (#1793) set and validate token audience on pds v2 * merge * include entryway did on tests * build branch * fix cache issue * xrpc server blob limit * put correct bytes * add auth to routes * handle quarantining/unquarantining a blob that does not exist * tidy * fix transfer tests * fix email request routes for entryway * PDS v2 entryway account deletion (#1819) * add admin lexicon for account deletion * implement admin account deletion endpoint * fix entryway proxying on account email checks * proxy to entryway for acct deletion * read-after-write sanity check * tweak * wip * finish refactor * fix test schema * application retry logic for busy * pr feedback * rm lru-cache * fix test pg schema * fix transfer test * Sqlite instrumentation for pds v2 (#1838) * sqlite instrumentation * build * remove build * dont reimport blobs * send ticks during import * close on error * catch handle validation error * add log * fix test * return emailConfirmedAt on getAccountInfo * Upgrade sharp on pds v2 (#1863) upgrade sharp to 0.32.6 * read all bytes before parsing car * Async car reader (#1867) * asynchronously read in car * dont buffer car * tweak * Gracefully handle indexing of invalid records (#1853) * gracefully handle indexing of invalid records * fix repo tests * Fix role auth for access-or-role verifier, getBlob check on actor takedowns (#1869) fix role auth for access-or-role verifier, fix getBlob actor takedown check * better cleanup of actor-stores * add ability to not ensure leaves * tidy * allow did:web transfer * Migration utility for actor-store (#1873) beginnings of helper for migrating all actors Co-authored-by: Devin Ivy <devinivy@gmail.com> * base case for findBlobRefs * App-level retries for sqlite on pds (#1871) * revamp retry helper to be more flexible re: backoff strategies * sqlite timeout helper * ensure sqlite wal on db creation/migration rather than every open * layer retries for sqlite on writes outside transactions on pds * tidy * fix up lockfile * tidy * fix lex codegen * fix timing bug in threadgate test * No-op update handling (#1916) do no produce commits on no-op updates * Retry on all SQLITE_BUSY error codes (#1917) retry on all sqlite_busy error codes * Pds v2 ensure sqlite ready (#1918) ensure sqlite is ready before making queries * try something * tidy * dont build branch --------- Co-authored-by: Devin Ivy <devinivy@gmail.com> |
||
|
Daniel Holmgren
|
fcb19c9c51 |
Simplify PDS moderation (#1723)
* spec out new simple pds mod routes * introduce new admin state endpoints * wire up routes * clean up pds * revoke refresh tokens * getUserAccountInfo * pr tidy * fixing some tests * fixing up more tests * fanout takedowns to pds * fanout admin reqs to pds * tidy * more tidy & add more pds moderation tests * getUserAccountInfo -> getAccountInfo * dont hydrate pds info on searchRepos * fix build * port admin tests to bsky package * clean up old snaps * tests on fanout * tweak naming * missed a rename * tidy renames * fix lex name * tidy & move snap * fix build * cleanup repeat process all * skip actor search test * fix bday paradox * tidy up pds service auth * rm skipped test * retry http * tidy * improve fanout error handling * fix test * return signing key in did-web * more tests * tidy serivce auth checks * change takedownId col to takedownRef * build branch * fix bsky test * add service key to indexer * move signing key to api entry * dont build |
||
|
Daniel Holmgren
|
44ea5e80fa |
Split apart auth grant & verification (#1743)
* transition to auth verifier * tidy * quick test fix |
||
|
Daniel Holmgren
|
d664b51c64 |
Finalize PDS in-process AppView removal (#1198)
* rm tables * rm event-stream & proxied * Remove appview services, move label service to pds * only proxy appview stuff * delete more tables * Start removing message dispatched from pds * more syncing-up removal of message dispatcher in pds * merged * remove feedgens from pds, remove getPopular * remove unused image helper from pds * fixing compiler errors * clean up sharp * rm label service * first pass on cleaning up tests * fix up a bunch of tests * moderation view tests * last admin tests * got a lil overzealous in deletes * clean up unused cfg * clean up label table * simplify admin repo search query/logic * tidy pds entrypoint * in-progress pds config changes * cfg fiddling * finish cleaning up cfg/ctx * comments * building * pds prefix on env * test env * collapse pds migrations down into a single migration * fix up dev-env * tidy * cleanup * fix pds admin tests * fix handle test * fix pds proxy tests * fix subscribe repos test * fix sqlite config in pds tests * add sqlite clause in sequencer-leader * fix actor search w/ sqlite on pds * fixes * fix dev env build * update pds service entrypoint * simple env example * make takedown ids opaque identifiers in the pds * use pds routes for api tests * update pds dockerfile with volume and correct port env var * add a couple env vars to example * add comments to env example * @atproto/pds 0.2.0-beta.0 * @atproto/aws 0.0.1-beta.0 * appview did * @atproto/aws 0.0.1 * enable logs by default * update env example * bugfixing sandbox issues * consistency in pds env var name for appview url * log on pds start and stop, configure version at runtime * @atproto/pds 0.2.0-beta.1 * fix semver matching for pds beta version * v0.2.0-beta.2 * default invites to being not required * fix flaky test * limit db connections in tests * publish 0.2.0-beta.d3 * fix invite required parsing * @atproto/pds 0.2.0-beta.5 * Proxy getPopularFeedGenerators on simplified pds (#1222) proxy getPopularFeedGenerators on pds Co-authored-by: dholms <dtholmgren@gmail.com> * tidy migrations * fix service entry * bump version * change auth order * bump version * bump version * add upgradeRepoVersion & fallback url for cdn * bump version * merging * merge pds * building dev-env * merging tests * merge service entry * test fixing * tidy * fix admin search * tidy * tidy * add snap for getListFeed * add backup nameserver cfg * tidy + pr feedback * tidy * tidy env * bit more * re-add dotenv to root package.json * fix dep * build branch * fix tests * Refactor tests to make better use of dev-env (#1690) * refactor pds tests to use dev env * refactor bsky tests * fix pds test * tidy bsky tests * build pds correctly * fix entry point * default logging to false (for now) * format service entry * Switch takedown ids back to ints on pds distribution (#1694) * switch takedown ids back to ints, consistent with live pds * tidy/fix migration * update migration for sqlite * export moderation action reversal * takedown tests * dont build branch --------- Co-authored-by: Devin Ivy <devinivy@gmail.com> |
||
|
Daniel Holmgren
|
41ee177f5a |
Email confirmation/update (#1568)
* lexicons * codegen * email templates * request routes * impl * migration * tidy * tests * tidy & bugfixes * format * fix api test * fix auth test * codegen * add unique constraint * Add email confirmed to AtpSessionData * interop test files (#1529) * initial interop-test-files * crypto: switch signature-fixtures.json to a symlink * syntax: test against interop files * prettier * Update interop-test-files/README.md Co-authored-by: Eric Bailey <git@esb.lol> * disable prettier on test vectors --------- Co-authored-by: Eric Bailey <git@esb.lol> Co-authored-by: dholms <dtholmgren@gmail.com> * add getSuggestedFollowsByActor (#1553) * add getSuggestedFollowsByActor lex * remove pagination * codegen * add pds route * add app view route * first pass at likes-based suggested actors, plus tests * format * backfill with suggested_follow table * combine actors queries * fall back to popular follows, handle backfill differently * revert seed change, update test * lower likes threshold * cleanup * remove todo * format * optimize queries * cover mute lists * clean up into pipeline steps * add changeset * List feeds (#1557) * lexicons for block lists * reorg blockset functionality into graph service, impl block/mute filtering * apply filterBlocksAndMutes() throughout appview except feeds * update local feeds to pass through cleanFeedSkeleton(), offload block/mute application * impl for grabbing block/mute details by did pair * refactor getActorInfos away, use actor service * experiment with moving getFeedGenerators over to a pipeline * move getPostThread over to a pipeline * move feeds over to pipelines * move suggestions and likes over to pipelines * move reposted-by, follows, followers over to pipelines, tidy author feed and post thread * remove old block/mute checks * unify post presentation logic * move profiles endpoints over to pipelines * tidy * tidy * misc fixes * unify some profile hydration/presentation in appview * profile detail, split hydration and presentation, misc fixes * unify feed hydration w/ profile hydration * unify hydration step for embeds, tidy application of labels * setup indexing of list-blocks in bsky appview * apply list-blocks, impl getListBlocks, tidy getList, tests * tidy * update pds proxy snaps * update pds proxy snaps * fix snap * make algos return feed items, save work in getFeed * misc changes, tidy * tidy * fix aturi import * lex * list purpose * lex gen * add route * add proxy route * seed client helpers * tests * mutes and blocks * proxy test * snapshot * hoist actors out of composeThread() * tidy * tidy * run ci on all prs * format * format * fix snap name * fix snapsh --------- Co-authored-by: Devin Ivy <devinivy@gmail.com> * Improve xrpc server error handling (#1597) improve xrpc server error handling * Remove appview proxy runtime flags (#1590) * remove appview proxy runtime flags * clean up proxy tests * getPopular hotfix (#1599) dont pass all params * Interaction Gating (#1561) * lexicons for block lists * reorg blockset functionality into graph service, impl block/mute filtering * apply filterBlocksAndMutes() throughout appview except feeds * update local feeds to pass through cleanFeedSkeleton(), offload block/mute application * impl for grabbing block/mute details by did pair * refactor getActorInfos away, use actor service * experiment with moving getFeedGenerators over to a pipeline * move getPostThread over to a pipeline * move feeds over to pipelines * move suggestions and likes over to pipelines * move reposted-by, follows, followers over to pipelines, tidy author feed and post thread * remove old block/mute checks * unify post presentation logic * move profiles endpoints over to pipelines * tidy * tidy * misc fixes * unify some profile hydration/presentation in appview * profile detail, split hydration and presentation, misc fixes * unify feed hydration w/ profile hydration * unify hydration step for embeds, tidy application of labels * setup indexing of list-blocks in bsky appview * apply list-blocks, impl getListBlocks, tidy getList, tests * tidy * update pds proxy snaps * update pds proxy snaps * fix snap * make algos return feed items, save work in getFeed * misc changes, tidy * tidy * fix aturi import * initial lexicons for interaction-gating * add interactions view to post views * codegen * model bad reply/interaction check state on posts * initial impl for checking bad reply or interaction on write * omit invalid interactions from post thread * support not-found list in interaction view * hydrate can-reply state on threads * present interaction views on posts * misc fixes, update snaps * tidy/reorg * tidy * split interaction gating into separate record in lexicon * switch interaction-gating impl to use separate record type * allow checking reply gate w/ root post deletion * fix * initial gating tests * tighten gated reply views, tests * reply-gating list rule tests * allow custom post rkeys within window * hoist actors out of composeThread() * tidy * update thread gate lexicons, codegen * lex fix * rename gate to threadgate in bsky, update views * lex fix * improve terminology around reply validation * fix down migration * remove thread gates on actor unindexing * add back .prettierignore * tidy * run ci on all prs * syntax * run ci on all prs * format * fix snap --------- Co-authored-by: Devin Ivy <devinivy@gmail.com> * order by `like.indexedAt` in app view (#1592) * order by like.indexedAt * use keyset for ordering * simplify * ok ok ok I get it now * Update packages/bsky/src/api/app/bsky/feed/getActorLikes.ts Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com> --------- Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com> * Remove default value for post table invalid attrs (#1601) remove default value for post table attrs * Version packages (#1602) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * update Bluesky PBLLC to PBC (Public Benefit Corporation) (#1600) * Temporarily disable filtering `invalidReplyRoot`s (#1609) temporarily disable invalidReplyRoot check * fix syntax docs (#1611) * Version packages (#1612) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Allow bypass on ratelimit ip (#1613) allow bypass on ratelimit ip * Write rate limits (#1578) * get rate limit ip correctly * add write rate-limits * Tweak createSession rate limit key (#1614) tweak create session rl key * Filter preferences for app passwords (#1626) filter preferences for app passwords * Tweak rate limit setup for multi rate limit routes (#1627) tweak rate limit setup for multi rate limit routes * Remove zod from xrpc-server error handling (#1631) remove zod from xrpc-server error handling check * Enforce properties field on lexicon object schemas (#1628) * add empty properites to thread gate schema fragments * tweak lexicon type * Add feed-vew and thread-view preferences (#1638) * Add feed and thread preference lexicons * Add feed-view and thread-view preference APIs * Add changeset for new preferences (#1639) Add changeset * Version packages (#1640) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Disable getAccountInviteCodes for app passwords (#1642) disable getAccountInviteCodes for app passwords * remove cruft packages (uri, nsid, identifier) (#1606) * remove @atproto/nsid (previously moved to syntax) * remove @atproto/uri (previously moved to syntax) * remove @atproto/identifier (previously moved to syntax) * bump lockfile to remove old packages --------- Co-authored-by: Eric Bailey <git@esb.lol> * api: update login/resumeSession examples in README (#1634) * api: update login/resumeSession examples in README * Update packages/api/README.md Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com> --------- Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com> * small syntax lints (#1646) * lint: remove unused imports and variables * lint: prefix unused args with '_' * eslint: skip no-explicit-any; ignore unused _var (prefix) * eslint: explicitly mark ignores for tricky cases * indicate that getPopular is deprecated (#1647) * indicate that getPopular is deprecated * codegen for deprecating getPopular * tidy up package.json and READMEs (#1649) * identity: README example and tidy * tidy up package metadata (package.json files) * updated README headers/stubs for several packages * crypto: longer README, with usage * syntax: tweak README * Apply suggestions from code review Co-authored-by: Eric Bailey <git@esb.lol> Co-authored-by: devin ivy <devinivy@gmail.com> --------- Co-authored-by: Eric Bailey <git@esb.lol> Co-authored-by: devin ivy <devinivy@gmail.com> * Improve the types of the thread and feed preferences APIs (#1653) * Improve the types of the thread and feed preferences APIs * Remove unused import * Add changeset * Version packages (#1654) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Disable pds appview routes (#1644) * wip * remove all canProxyReadc * finish cleanup * clean up tests * fix up tests * fix api tests * fix build * fix compression test * update image tests * fix dev envs * build branch * fix service file * re-enable getPopular * format * rm unused sharp code * dont build branch * auto-moderator tweaks: pass along record URI, create report for takedown action (#1643) * auto-moderator: include record URI in abyss requests * auto-moderator: log attempt at hard takedown; create report as well The motivation is to flag the event to mod team, and to make it easier to confirm that takedown took place. * auto-mod: typo fix * auto-mod: bugfixes * bsky: always create auto-mod report locally, not pushAgent (if possible) * bsky: fix auto-mod build * bsky: URL-encode scanBlob call * Clear follow viewer state when blocking (#1659) * clear follow viewer state when blocking * tidy * add `tags` to posts (#1637) * add tags to post lex * kiss * add richtext facet and validation attrs * add tag validation attrs to post * codegen * add maxLength for tags, add description * validate post tags on write * add test * handle tags in indexer * add tags to postView, codegen * return tags on post thread view * format * revert formatting change to docs * use establish validation pattern * add changeset (cherry picked from commit fcb6fe7c26144662f791c7900afcd84c7bf1be6b) * remove tags from postView, codegen * remove tags from thread view * revert unused changes * Version packages (#1664) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * merge * Reverse order of blocks from sync.getRepo (#1665) * reverse order of blocks from sync.getRepo * write to car while fetching next page * Add hashtag detection to richtext (#1651) * add tag detection to richtext * fix duplicate tag index error * add utils * fix leading space index failures, test for them * add changeset * Version packages (#1669) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * proposed new search lexicons (#1594) * proposed new search lexicons * lexicons: lint * lexicons: fix actors typo * lexicons: camelCase bites again, ssssss * lexicons: add 'q' and mark 'term' as deprecated for search endpoints * codegen for search lexicon updates * bsky: prefer 'q' over 'term' in existing search endpoints * search: bugfix * lexicons: make unspecced search endpoints return skeleton obj * re-codegen for search skeleton obj * Disable pds appview indexing (#1645) * rm indexing service * remove message queue & refactor background queue * wip * remove all canProxyReadc * finish cleanup * clean up tests * fix up tests * fix api tests * fix build * fix compression test * update image tests * fix dev envs * build branch * wip - removing labeler * fix service file * remove kysely tables * re-enable getPopular * format * cleaning up tests * rm unused sharp code * rm pds build * clean up tests * fix build * fix build * migration * tidy * build branch * tidy * build branch * small tidy * dont build * Refactor PDS appview routes (#1673) move routes around * Strip leading `#` from from detected tag facets (#1674) ensure # is removed from facets * Version packages (#1675) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * Proxy search queries (#1676) * proxy search * tweak profile resp * fix admin.searchRepos * add mock mailer * Fix to daniel's MOCKERY of a mock mailer * Don't allow non-verified email updates until app feature is out (#1682) stricter updating email until app feature is out * changesets --------- Co-authored-by: Paul Frazee <pfrazee@gmail.com> Co-authored-by: bnewbold <bnewbold@robocracy.org> Co-authored-by: Eric Bailey <git@esb.lol> Co-authored-by: Devin Ivy <devinivy@gmail.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> |
||
|
Daniel Holmgren
|
6446e8d1d6 |
App passwords (#826)
* app password lex & auth chnages * scrypt things * implemented app password refresh tokens * db tidy & migration * revocation + bugfixin * tests, listing passwords & cleanup * Update packages/pds/src/db/scrypt.ts Co-authored-by: devin ivy <devinivy@gmail.com> * Update packages/pds/src/db/scrypt.ts Co-authored-by: devin ivy <devinivy@gmail.com> * pr feedback --------- Co-authored-by: devin ivy <devinivy@gmail.com> |
||
|
Daniel Holmgren
|
fc4a34caf2 |
Fully deprecate handle on createSession (#761)
* fully deprecate handle on createSession * fix tests |
||
|
Daniel Holmgren
|
095652b75b |
Include email on server.getSession (#756)
* include email on getSession * fix up tests & return email on createSession * fix more tests * cleanup |
||
|
Daniel Holmgren
|
7f008c05a0 |
Lexicon refactor (#658)
* remove return in test * couple of fixups in other pacakges * Add dummy checks to declaration and follow app migrations, remove paranoid join * update db nsid migration * Ensure there are writes in follow app migration * Add dumy check to votes-to-likes app migration, tidy * Ensure there are writes in vote-to-like app migration * update migration name * pr feedback * count utf8 & grapheme length * add maxUtf8 * siwtch max semantics * plural * update post schema * added bytes & cid refs * add ipld<>json * fixin up a could tings * Add app.bsky.richtext.facet, replace post entities with facets * plural actors * wip * Setup backlinks table on pds * wip * send & recieve cids/bytes with xrpc * Track backlinks when indexing records on pds * handle ipld vals in xrpc server * added cids & bytes to codegen * In createRecord, add deletions to avoid duplicate likes/follows/reposts * Tests and fixes for prevention of dupe follows, likes, reposts * Backlink migration tidy * cleanup dag json parser * Fix dupe backlink inserts * Tidy * blob refs + codegen * Make profile displayName optional * Test view and updateProfile for empty display name * working into pds * Make aggregate counts optional on post and profile views * Make viewer state optional on post view for consistency * Remove deprecated myState field on profile view * Tidy repo method descriptions * tests & types & fixes * Implementation and tests for putRecord * Remove updateProfile method * Update repo service so that head can be taken for update externally * Lex updates for compare-and-swap records/commits * Add error to lex for bad repo compare-and-swaps * Improve update-at-head thru repo service * common package * Implement and test compare-and-swaps on repo write methods * Use lex discriminator for applyWrites * Remove post entity/facet index * Update lex descriptions to clarify repo write semantics * Make deleteRecord idempotent w/ tests * cleanup * fix things up * adding more formats * tests * updating schema * Only generate tid rkeys on pds, support literal rkeys on client * Add backlink indexes * Update format of post embed views, fix external uri validation * fixing up tests * Include embeds on record embeds * cleanup * Notify users when they are quoted * Remove determineRkey indirection * fix api tests * support concatenated cbor * integrating to server * re-enable tests * fix up tests * Thread compare-and-swaps down into repo service rather than use pinned storage * Tidy * Update packages/common/tests/ipld-multi.test.ts Co-authored-by: devin ivy <devinivy@gmail.com> * Update packages/lexicon/src/validators/formats.ts Co-authored-by: devin ivy <devinivy@gmail.com> * pr feedback * pr feedback * Add postgres-specific migration path for missing profile display names * Tidy/clarify deep embeds * Tidy * rm unused escape * decrease crud race count * update subscribeRepos lexicon * Fix applyWrite lexicon re: collection fields * sign post event type * update cids & bytes json encoding * update lex blob & cid-link types * updated codegen & pds * number -> float * missed a couple * remove old image constraints * pr feedback + descripts * no hardcoded port numbers * remove separate tooLarge evt * fix dumb build error * fixin gup lex + xrpc server * better parsing of message types * dont mutate body in subscription * bugfix in subscription * rm commented out code * init feature branch * undo * Remove old lexicons * Remove creator from profile view * wip * rework seqs * fixed up tests * bug fixing * sequence handles & notify in dbTxn * tidy * update lex to include times * test syncing handle changes * one more fix * handle too big evts * dont thread sequencer through everything * Split common into server vs web-friendly versions * Make lexicon, identifier web-safe using common-web * Switch api package to be a browser build, fix identifier package for browser bundling * Fix pds and repo for lexicon package changes, tidy * Make common-web a browser build, tidy * fixing up deps * fix up test * turn off caching in actions * Standardize repo write interfaces around repo input * Update repo write endpoints for repo input field * Remove scene follows during app migration * API package updates (#712) * Add bsky agent and various sugars to the api package * Add richtext library to api package * Update richtext to use facets and deprecate entities * Update richtext to use utf8 indices * Richtext converts deprecated entity indices from utf16 locations to utf8 locations * Add note about encodings in the lexicon * Add RichText facet detection * Remove dead code * Add deprecation notices to lexicons * Usability improvements to RichText * Update the api package readme * Add RichText#detectFacetsWithoutResolution * Add upsertProfile to bsky-agent * Update packages/pds/src/api/com/atproto/repo/applyWrites.ts Co-authored-by: devin ivy <devinivy@gmail.com> * pr feedback * fix flaky timing streaming tests * simplify emptyPromise * fixed up open handles * fix missed repo syntax * fix error in test from fkey constraint * fix another api agent bug * Embed consistency, add complex record embed * Tidy embed lex descriptions * rename pg schemas * use swc for jest * fix up deps * cleanup * Update pds indexing, views, tests for complex record embeds * fixing up profile view semantics * wip * update snaps * Rename embed.complexRecord to embed.recordWithMedia * Tidy aroud record w/ media embeds * Add grapheme utilities to api RichText (#720) Co-authored-by: dholms <dtholmgren@gmail.com> * Fix: app.bsky.feed.getPostThread#... to app.bsky.feed.defs#... (#726) * Update bskyagent to use repo param * Minor typing fix * Add exports to api package: blobref & lex/json converters (#727) * Add exports to api package: BlobRef & lex/json converters * Add an example react-native fetch handler * Switch all lingering references of recordRef to strongRef * Update lexicon for richtext facets to have multiple features, byte slice rather than text slice * Implement multi-feature richtext facets on pds * Update api package to use updated richtext facets * Minor fixes to admin repo/record views * Fix app migration exports, remove old app migration * Fix: sort richtext facets so they can render correctly * Disable app migration dummy checks that don't work on live deploy * Optimize lex de/serialization using simple checks * Tidy comment typos * App migration to cleanup notifications for likes, follows, old scene notifs * Fix notification reason for change from vote to like --------- Co-authored-by: Devin Ivy <devinivy@gmail.com> Co-authored-by: Paul Frazee <pfrazee@gmail.com> |
||
devin ivy
|
30a3529c8c |
PDS refresh token grace period (#657)
* Add grace period to pds refresh tokens * Tidy * Tidy |
||
Paul Frazee
|
2242e8a313 |
Refactor @atproto/api to the AtpAgent interface (#529)
* Refactor @atproto/api to use the simplified AtpAgent API * xrpc package: Export the defaultFetchHandler to reuse in api * api package: Use the defaultFetchHandler defined in xrpc * Update all usages of the api for the new AtpAgent * Clear promise on thrown codepath * Avoid updating the atpagent session until ready to return |
||
|
devin ivy
|
9834c2f5ef |
Allow login using email (#492)
* Allow login using email, deprecate handle input in session.create * Remove handle input from session.create lexicon |
||
|
devin ivy
|
ef487900f6 |
Server moderation action/resolution and reporting (#458)
* Model mod actions for actor takedowns * Add administration lex schemas for moderation actions * Initial implementation of takedown mod action and reversal * Refactor admin actions into a service * Check for account takedown when creating new session or refreshing session * Sweep usage of getUser() and getUserByEmail() for takedowns * Begin sweep for account takedowns * Tidy takedown checks * Handle post author takedowns * Fix tests and migrations for mod takedown * Fix xrpc-server delayed read of blob payloads * Test takedown mod action, misc fixes * Remodel lex schemas for admin moderation actions * Remove delcaration cid from actor takedowns, lex schema tidy * Add notes * Move moderation methods to com.atproto, based around repo rather than actor * Moderation reports of repos and records * Implement moderation report resolution method * Tidy * Place moderation-init migration after mute-init * Allow services to reference each other * Model record takedowns in db * Lex schemas for record takedowns * Service/methods for record takedowns, fix report view subject * Apply record takedown to posts in feeds * Test record takedown effects, fix reversal * Support server moderation flag and acknowledge actions, fix action/report resolution for records * Track repo takedowns on repo_root rather than did_handle * Move repo/record reporting under com.atproto.report * Colocate all moderation functionality into moderation service, combine from admin and repo services * Tidy * Tidy * Unify mod report and action methods with new repo/record ref schemas |
||
|
Daniel Holmgren
|
f0ee51f151 |
App context (#424)
* introducing appcontext & pds class * revamp rest of server + auth * revamp tests * add to dev-env * fix api test * add start method to pds * hook up terminator before * PLC AppContext (#425) * convert plc server to app context * add start method to plc * hooke up terminator on server setup * fix test * cleanup & pr feedback * fix up one more test |
||
|
Daniel Holmgren
|
7425fba702 |
Bad password test (#418)
quick test for bad pass |
||
|
Paul Frazee
|
63b9873bb1 |
Lexicon rework (#379)
* Implement types and validation for new NIH lexicon DSL * Update xprc and xrpc-server to use new NIH lexicon DSL * Simplification: dont allow lexicon definitions to be refs or ref arrays * Add 'unknown' primitive type to NIH lexicon * Implement client codegen for NIH Lexicon * Implement server codegen for NIH Lexicon * Add the 'datetime' primitive type to the NIH Lexicon * Rework lexicons to use new NIH Lexicon DSL * Readd tokens to code generation * Update all code to use new NIH Lexicon (all tests passing!) * Add imports from external files to lex-cli codegen * Do not allow records, queries, or procedures outside of the main def * Update packages/lex-cli/src/util.ts Co-authored-by: devin ivy <devinivy@gmail.com> * Linter pass * Add knownValue output to codegen * Add comments to codegen * Use enum and const in codegen * Rework lexicon 'properties' structure * Treat falsy query params as undefined Co-authored-by: devin ivy <devinivy@gmail.com> * Remove now-redundant param presence validation Co-authored-by: devin ivy <devinivy@gmail.com> |
||
|
Paul Frazee
|
e61127111a |
Split lexicons into more specific namespaces (#306)
* Segment lexicons into more specific namespaces and tighten up some naming conventions * Update API to use new lexicons * Update pds and dev-env to use new lexicons |
||
|
Daniel Holmgren
|
7fdbb4dbb5 |
Username -> Handle (#303)
* schemas * db * atproto xrpc * bsky xrpc * got er done |
||
|
Paul Frazee
|
c986066e4c |
Update XRPC procedures to use request body by default (#297)
* Update lexicons of xprc procedures to avoid using query params when possible * Update code generated by lex-cli to reduce param counts * Update all API usages to new signatures * Fix readme * Remove params from postNotifiactionsSeen lexicon schema |
||
|
devin ivy
|
51dd5665a1 |
Session refresh token for PDS (#286)
* Update lexicons for auth refresh token * Implement pds refresh tokens * Test pds auth sessions * Fix dev-env * Tidy * Ignore invalid revoked and expired tokens during deleteSession |