* tidy bsky auth
* hook up new auth verifier
* update auth throughout ozone
* handle mod signing keys
* fix bad var
* fix key parsing in pds
* fix admin auth test
* rename test
* update did doc id values
* null creds string -> `none`
* fix fetchLabels auth check
* ✨ Add a couple more proxied requests that we use in ozone ui
* Add runit to the services/bsky Dockerfile (#2254)
add runit to the services/bsky Dockerfile
* Improve tag detection (#2260)
* Allow tags to lead with and contain only numbers
* Break tags on other whitespace characters
* Export regexes from rich text detection
* Add test
* Add test
* Disallow number-only tags
* Avoid combining enclosing screen chars
* Allow full-width number sign
* Clarify tests
* Fix punctuation edge case
* Reorder
* Simplify, add another test
* Another test, comment
* Version packages (#2261)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* 🐛 Increment attempt count after each attempt to push ozone event (#2239)
* Ozone delegates email sending to actor's pds (#2272)
* ozone delegates email sending to user's pds
* lexicon: add content field to mod email event
* test email sending via mod event
* fix auth verifier method
* better error handling for get account infos
* fix labeler service id
* fix iss on auth headers
* fix dev-env ozone did
* fix tests & another jwt issuer
* ozone: fix ip check
* fix aud check on pds mod service auth
* tidy
---------
Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
Co-authored-by: Jake Gold <52801504+Jacob2161@users.noreply.github.com>
Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: devin ivy <devinivy@gmail.com>
* Version packages
* Give Mary credit
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Eric Bailey <git@esb.lol>
* cache did docs in redis
* drop table
* expire from redis
* fix tests
* add cache class
* update api
* refactor
* filter negative labels
* fix up dev-env
* refactor did cache to use new redis cache class
* tidy
* ensure caching negatives
* redis cache tests
* remove timeout on did cache
* fix ns in test
* rename driver
* add timeout & fail open
* add test for timeout & fail open
* add basic rate limits
* tidy
* small pr feedback
* refactor caches
* bugfixg
* test for caching negative values
* little more to cache
* wire up cache cfg
* switch from redis scratch to redis
* fix build issues
* use different redis clients for tests
* fix test
* fix flaky test
* fix build issue
* use separate db for redis cache
* cleanup repeat process all
* wip
* skip actor search test
* skip actor search test
* tweak processAll
* decrease wait to 1 sec
* repo_blob -> record_blob
* simplify backlink linkTo
* return repo_root to one row
* sequence before updating repo_root
* invite code forUser -> forAccount
* ipld_block -> repo_block
* use lru-cache fetchMethod
* move did_cache to own db
* better error handling on did cache
* drop did_handle
* fix sequencer wait time
* debug
* debug
* more debug
* check something
* fix bday paradox
* fix bday paradox
* tidy up pds service auth
* rm skipped test
* retry http
* tidy
* improve fanout error handling
* fix test
* return signing key in did-web
* more tests
* tidy serivce auth checks
* user_account -> account
* remove inviteNote
* keypair per repo
* use an lru cache for keypairs as well
* clean up repo
* wip
* wrap up accoutn manager
* tidy
* tidy
* fix tests
* fix disabled codes
* fix appview tests
* add note
* set pragmas
* tidy account manager getDb
* rename pref transactor
* user pref -> account pref
* handle blob imports
* tidy imports
* add reserveSigningKey
* wip transferAccount
* clean up transferAccount
* tests
* tidy
* tidy
* configure entryway url on pds
* handle entryway in pds admin endpoints
* make importRepo temp
* fix imports
* make email optional on pds when using entryway
* handle diffs
* handle pds entryway usage for server, identity, admin endpoints
* pds support for credentials from entryway
* setup pds tests w/ entryway service
* tidy
* tidy
* update entryway version
* wip
* test handle updates w/ entryway
* split account table into two
* tidy
* tweak scripts
* tidy tests
* tidy
* better config for actorstore & dbs
* clean up cfg more
* reorg actorstore fs layout
* handle erros on actor db create
* pr tidy & fix accoutn deletion test
* pr feedback
* fix bad merge
* unskip test
* fix subscribe repos tests
* tidy repo root tables
* tidy
* fix tests
* tidy delete tokens
* tidy account getters
* tidy
* bulk deletesg
* increase chunk size
* handle racing refreshes
* wip
* fix auth test
* invert import flow
* clean up actor store on create account failure
* tweak sequencer
* prevent invite code races on createAccount
* rm note
* add back in race protection on getAccountInviteCodes
* start feature branch
* deleted app migration table
* patch up new auth test
* rm note
* g
* create accoutn delegated from entryway
* tidy
* fix test
* change plcOp type to unknown
* small fixes
* sync up w entryway branch
* Use proper error when authed account is not found (#1799)
provide proper error when account not found in access-takedown check
* build branch
* build on ghcr
* tweak service file
* tweak service file
* change where we save reserved keys
* no tmp dir in blobstore either
* fix blobstore temp location again
* handle repeat record_blobs
* create account before submitting plc op & undo if fail
* small tweak
* limit the number of local records
* push out empty commit on transfer
* fix issue with record_blob
* add push blob endpoint
* Set and validate token audiences on pds v2 (#1793)
set and validate token audience on pds v2
* merge
* include entryway did on tests
* build branch
* fix cache issue
* xrpc server blob limit
* put correct bytes
* add auth to routes
* handle quarantining/unquarantining a blob that does not exist
* tidy
* fix transfer tests
* fix email request routes for entryway
* PDS v2 entryway account deletion (#1819)
* add admin lexicon for account deletion
* implement admin account deletion endpoint
* fix entryway proxying on account email checks
* proxy to entryway for acct deletion
* read-after-write sanity check
* tweak
* wip
* finish refactor
* fix test schema
* application retry logic for busy
* pr feedback
* rm lru-cache
* fix test pg schema
* fix transfer test
* Sqlite instrumentation for pds v2 (#1838)
* sqlite instrumentation
* build
* remove build
* dont reimport blobs
* send ticks during import
* close on error
* catch handle validation error
* add log
* fix test
* return emailConfirmedAt on getAccountInfo
* Upgrade sharp on pds v2 (#1863)
upgrade sharp to 0.32.6
* read all bytes before parsing car
* Async car reader (#1867)
* asynchronously read in car
* dont buffer car
* tweak
* Gracefully handle indexing of invalid records (#1853)
* gracefully handle indexing of invalid records
* fix repo tests
* Fix role auth for access-or-role verifier, getBlob check on actor takedowns (#1869)
fix role auth for access-or-role verifier, fix getBlob actor takedown check
* better cleanup of actor-stores
* add ability to not ensure leaves
* tidy
* allow did:web transfer
* Migration utility for actor-store (#1873)
beginnings of helper for migrating all actors
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* base case for findBlobRefs
* App-level retries for sqlite on pds (#1871)
* revamp retry helper to be more flexible re: backoff strategies
* sqlite timeout helper
* ensure sqlite wal on db creation/migration rather than every open
* layer retries for sqlite on writes outside transactions on pds
* tidy
* fix up lockfile
* tidy
* fix lex codegen
* fix timing bug in threadgate test
* No-op update handling (#1916)
do no produce commits on no-op updates
* Retry on all SQLITE_BUSY error codes (#1917)
retry on all sqlite_busy error codes
* Pds v2 ensure sqlite ready (#1918)
ensure sqlite is ready before making queries
* try something
* tidy
* dont build branch
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* bust key cache when verifying service auth
* unit tests for xrpc auth
* fix
* support option for verifying non-low-s signatures
* fix verifyJwt tests
* spec out new simple pds mod routes
* introduce new admin state endpoints
* wire up routes
* clean up pds
* revoke refresh tokens
* getUserAccountInfo
* pr tidy
* fixing some tests
* fixing up more tests
* fanout takedowns to pds
* fanout admin reqs to pds
* tidy
* more tidy & add more pds moderation tests
* getUserAccountInfo -> getAccountInfo
* dont hydrate pds info on searchRepos
* fix build
* port admin tests to bsky package
* clean up old snaps
* tests on fanout
* tweak naming
* missed a rename
* tidy renames
* fix lex name
* tidy & move snap
* fix build
* cleanup repeat process all
* skip actor search test
* fix bday paradox
* tidy up pds service auth
* rm skipped test
* retry http
* tidy
* improve fanout error handling
* fix test
* return signing key in did-web
* more tests
* tidy serivce auth checks
* change takedownId col to takedownRef
* build branch
* fix bsky test
* add service key to indexer
* move signing key to api entry
* dont build
* use pnpm
* fix dependency issues, replace yarn and lerna scripts
* remove the main/dist scripts
* update Dockerfiles
* use pnpm
* fix dependency issues, replace yarn and lerna scripts
* remove the main/dist scripts
* update Dockerfiles
* update bin script
* remove unused zod dep
* fix type errors in pds
* add types prop to packages
* remove unused, bump lock
* fix test running
* build before test
* fix pino types
* format
* pds depends on dev-env in test
* refer to src instead of built packages
* pds relies on bsky in test too
* remove yarn.lock
* add -r flag to root test
* test push to aws
* remove docker test
* add publishConfig to new package
* move services to top level dir
(cherry picked from commit f5012bec33435a4473e9960066807623334f3aff)
* update workflow paths
(cherry picked from commit 5c70f0176d381ca35d6be10cfa173e22373a5b5d)
* add main-to-dist script
* use script in all packages, remove old Dockerfiles
* remove old bsky service
* remove newline
* test container builds
* Revert "test container builds"
This reverts commit c228611f5e8e1624d4b124be4976c49590130f43.
* remove unused config
* test build containers
* pnpm in syntax
* bump dd-trace
* shamefully hoist
* even more shame
* hoist, externalize deps
* clean install for prod and smaller containers
* dont build branches
---------
Co-authored-by: dholms <dtholmgren@gmail.com>
* Model post and profile aggs in bsky
* Add background, onCommit, db and background stats
* Add aggregations for post and profile
* Use background queue in bsky labeler, misc plumbing
* Process background throughout bsky tests
* bsky tidy and fixes
* Build views using agg tables in bsky
* Views of feedgens on bsky
* Setup custom feeds on bsky
* Feed views refactor for bsky, implement getFeed w/ auth passthrough
* Passthrough auth from bsky appview to feedgen, fixes/tidy/tests
* Test custom algos on bsky
* Test pds getFeed proxy
* tidy, fix build
* tidy
* nsid: reduce 'name' length from 128 chars to 63 chars
* nsid: remove 'nsid-ns' (glob pattern) from main NSID syntax validation
This special variant syntax should be handled separately. Eg, an
"nsid-ns" should not be accepted as an NSID in Lexicons looking for a
regular NSID.
* nsid: update syntax check to match domain rules
This is to be closer to actual underlying specification needs. For
example, domains are allowed to start with a digit (like 4chan.org), and
can't start *or* end with a hyphen.
Restricts the "name" part further, to be alphabetic only.
Note that starts-with-digit domains could break some lang
auto-generation variable name rules. The docs/spec strongly recommend
against such domains for use with NSID, but it seems incorrect to
restrict at the standard/lexicon level without a clear pan-language
consistent set of rules.
* did-resolver: disallow did:web with path segments
* identifier: reduce max length of DID from 8 KB to 2 KB
* identifier: add DID test with escaped ':' in identifier
* did-resolver: have prettier run on tests/
* identifier: add additional punycode test cases
* identifiers: small tweaks from review
- type in did:web comment
- include actual max DID string length in error message
* xrpc-server tests: remove digits from NSIDs
* identity: fix import and function errors
* xrpc-server tests: remove digits from NSIDs
* xrpc-server: include tests/ in prettier:fix
* xrpc-server: lint fix
* Better feedgen errors in getFeed and getFeedGenerator
* Support handler output headers in codegen
* Support handler output headers in xrpc-server
* Implement server-timing helpers and apply them to getFeed
* fix codegen issue
---------
Co-authored-by: dholms <dtholmgren@gmail.com>
* Return `XRPCError` instead of 404 if a handler errors
Currently, if there is an uncaught error inside a handler, a 404 response is returned instead of the error itself. This fixes that.
* Catch it in the higher-level catch block
* Add test
* merge
* Improve test
Matthieu Sieben
github-actions[bot]
Daniel Holmgren
dan
bnewbold
devin ivy
Eric Bailey
Patrick Linnane
Wes Todd
Roj
Wes Bos