Commit Graph

2335 Commits

Author SHA1 Message Date
Patryk a7c2b4ae11 Add block record duplicate detection (#1027)
* Add block record duplicate detection

* Update packages/pds/src/services/record/index.ts

---------

Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>
2023-05-11 11:00:24 -04:00
devin ivy 5f0db0c844 Update zod version, some perf improvements (#1021)
* Update zod version, some perf improvements

* Fix circular type ref
2023-05-11 10:57:12 -04:00
dholms 3783889c49 bump @atproto/did-resolver 2023-05-10 23:24:33 -05:00
dholms 2f43fb7454 bump api package version 2023-05-10 20:50:56 -05:00
bnewbold 7368020068 pds: place-holder robots.txt, and a text base-level route (/) (#995)
* pds: place-holder robots.txt and / (as txt)

robots.txt: want to be explicit about allowing crawling (generically)

/: a bit of dev-experience polish. in prod we already redirect this
route, but may be helpful for self-hosting folks and people just
exploring

* pds: move basic routes (health, robots.txt, /) to separate file
2023-05-10 17:08:52 -07:00
Patryk 0e35bc3899 Make BskyAgent instantiation 4 times faster (#1018)
Improve BskyAgent performance

By using z.discriminatedUnion instead of z.union
we can hint to zod at which field it should look when
validating/parsing schemas

This improves BskyAgent instantiation time around 4 times,
when creating 10 agents using stuff from #1015,
the time went down from ~1200ms to 330ms
2023-05-10 12:22:58 -04:00
Daniel Holmgren 89b783eec9 Update how sequencer invalidation works (#1014)
* invalidatedBy -> invalidated

* fix test

* dont remake fkey constraint sqlite

* sqlite migration

* appeasing sqlite

* sqlite finally happy?

* Fix sqlite migration, run proxy tests thru postgres when configured

* Fix pg schemas in proxy tests

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-05-10 11:01:56 -04:00
bnewbold a449d793a6 disallow new registration of .arpa, .local, etc handles (#1016)
* handles: test 'dotless' disallowed; move .arpa/.local proto-level test

* handles: disallow infra/reserved handles
2023-05-09 16:35:06 -07:00
devin ivy 977202fa2b Fix indexed record datetimes (#1009)
* Normalize record datetimes used for sort to simplified iso

* Update tests and snapshots for alternate valid datetime formats

* Fix notifications race during record update
2023-05-09 12:53:33 -04:00
devin ivy c3c75322bf Sync-up account deletion indexes migration (#1006) 2023-05-09 12:52:56 -04:00
Daniel Holmgren d30b0f93e1 Disable account invites (#1004)
* enable disabling account invites

* xtra test

* couple fixes

* allow re-enabling user accoutn invites

* add in getter route for if invites are disabled or not

* fix admin search test

* gneerated type on user account
2023-05-09 09:24:12 -05:00
Daniel Holmgren 51503384b6 No cache invalidation on blob deletion (#1008)
no invalidate on blob deletion
2023-05-09 08:57:45 -05:00
Daniel Holmgren b20a5045a2 Rebase route (#948)
* tidy rebase query

* add rebase route

* bring tests up to speed

* add tx check
2023-05-09 05:54:48 -05:00
Daniel Holmgren c8e6adbbf5 Clean up what's hot (#1002)
* changes to clean up whats hot

* fix test

* add quote posts back in & use underwear label instead of no-promote

* tidy
2023-05-08 18:26:11 -05:00
devin ivy 41793b7ff9 Add indexes to support account deletion (#1003)
* Add indexes to support account deletion

* Temporarily skip index creation in migration
2023-05-08 18:47:25 -04:00
Jaga Santagostino 652eb50da6 Update README.md example (#993)
Update README.md
2023-05-08 17:24:01 -05:00
Wes Todd b01e47b617 fix: updated repository field for packages (#994) 2023-05-08 11:38:42 -04:00
Daniel Holmgren aabbf43a7f Blessed hell thread (#988)
give notifs in blessed hell thread
2023-05-05 20:51:57 -05:00
Daniel Holmgren 8e5f2bd2dc Do not give deeply nested notifs (#987)
do not give deeply nested notifs
2023-05-05 19:00:02 -05:00
Daniel Holmgren bfe59e2bd3 Do not filter on negative labels on getPopular (#985)
do not filter on negative labels on whats hot
2023-05-05 18:59:49 -05:00
bnewbold 8c19ce991a moderation action 'escalate' type (#976)
* lex: 'escalate' mod action type

* api: codegen for escalate mod action type

* pds: codegen+impl esclate mod action type

* bsky: codegen+impl esclate mod action type

* pds: prettier:fix

* Tidy

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-05-04 12:32:27 -07:00
Daniel Holmgren b75436b9cb Delete user state on account delete (#970)
* Delete user state on account delete

* blocks too
2023-05-03 22:39:57 -05:00
Daniel Holmgren 015fe1226c Delete dereferenced blobs (#959)
* delete dereferenced blobs

* invalidate cache for delete blobs

* start testing

* test & cleanup

* rm blob cleanup logic on rebases

* make disk blob deletes idempotent

* move blob deletion to background queue
2023-05-03 18:34:55 -05:00
Daniel Holmgren b182344fef Avoid sequencer retry loop (#958)
* avoid seq loop

* tidy

* comments
2023-05-02 17:07:18 -05:00
devin ivy a1666776b3 Log requests to subscribeRepos (#957) 2023-05-02 17:37:23 -04:00
Daniel Holmgren 4555596212 Filter server-side labels (#956)
filter server side labels
2023-05-02 16:00:41 -05:00
devin ivy b1493650f4 Perform account deletion async (#951)
* Perform takedown then background hard-deletion process for deleteAccount

* Remove transactions from account deletion process, perform steps sequentially
2023-05-02 14:24:59 -04:00
devin ivy 142c77cadd Disallow writes of bad records (#955)
* Disallow writes of bad records

* Increase obj depth to hit stack issue on more systems
2023-05-02 13:44:07 -04:00
Mustafa Yafes Akbaş dc51cafa3d Fix SSRF vulnerability on blob (#893)
* Add hostname and protocol validation for pds in getBlob()

Hostname and protocol validation for the pds' url in the getBlob() to prevent potential SSRF (Server-Side Request Forgery) attacks by ensuring that only valid protocols and hostnames are used in the URL. Function also will now throw a 400 error if detects any invalid.

* Migrate changes

* delete the line with console.log...
2023-05-02 11:51:25 -05:00
Daniel Holmgren 86fae264d3 Sequencer race condition (#906)
* wip

* split out seq invalidations

* wip

* fix

* comments

* 2 db hits

* cleanup

* re-add comment
2023-05-02 11:39:25 -05:00
Roj fa95f5b20f Return InternalServerError instead of XRPCNotSupported when a falsy value is thrown (#894)
* Return `XRPCError` instead of 404 if a handler errors

Currently, if there is an uncaught error inside a handler, a 404 response is returned instead of the error itself. This fixes that.

* Catch it in the higher-level catch block

* Add test

* merge

* Improve test
2023-05-02 11:39:07 -05:00
Ryan Winchester 5333258771 Fix NSID name length error message typo (#918)
The name length limit is 128 not 127.
2023-05-02 11:24:30 -05:00
Max Graey 7329c28564 Some improvements for tid.ts (#936)
* Some improvements for tid.ts

* refactor dedash helper

* tripple equal

* run prettify
2023-05-02 10:53:19 -05:00
bnewbold 5fd71c099b appview: tweak repo error log line to not clobber 'service' (#891)
Somehow the output of this log line has been ending up in datadog under
the wrong 'service' (with the 'service' being an "wss://"-prefixed URL,
not "pds" or "bav" or whatever expected).

Tweaks the 'service' key, and some others that might have conflict.
2023-05-01 20:53:38 -05:00
Wes Bos 958c9ef6b3 add repository field to package.json (#895) 2023-05-01 20:43:32 -05:00
Roj 3370eac6ca Improve readability in instanceof assertions (#909)
* Improve readability in `instanceof` assertions

* Remove unused import
2023-05-01 20:43:09 -05:00
Daniel Holmgren 12c6d886eb Deterministic DNS handle resolution (#883)
* enforce constraint for one handle per domain

* tweak + tests
2023-05-01 20:27:12 -05:00
Daniel Holmgren 1e24d3e820 Include nsfw flag for getPopular (#949)
add includeNsfw flag for whats hot
2023-05-01 15:50:21 -05:00
Daniel Holmgren bff96544d1 Tweak whats hot (#945)
* tweak whats hot

* couple fixes & test
2023-05-01 14:10:48 -05:00
bnewbold 0b70f50eb4 pds: iterated Hive label parsing (#941)
Tune "porn", "sexual", and "nudity" label categorization, including
re-structuring of code paths.

This comes out of some image testing as well as review of Hive API docs.
2023-05-01 11:20:50 -07:00
devin ivy 9366734f6d More aggressive backpressure on websocket writer (#929)
* Implement more aggressive backpressure on websocket writer, may be temporary

* Add heap snapshot signal to pds
2023-05-01 08:48:45 -04:00
Devin Ivy 3d1b3b3675 Add csp and nosniff when serving user uploads 2023-04-29 11:47:50 -04:00
Paul Frazee 01b5971907 api@0.2.11, pds@0.1.5 (#925)
* @atproto/api@0.2.11

* @atproto/pds@0.1.5
2023-04-28 19:41:01 -05:00
Daniel Holmgren db1042cf0d Blocks (#922)
* block lexicons

* table schema

* block processing

* Actor-block migration

* Add blocked post lexicon to getPostThread

* getBlocks routes

* Add missing lexicons to bsky

* Add block state to actor views

* throw on blocked accounts in getAuthorFeed

* blcok state in profile views

* Update getPostThread to support block views, update existing test

* strip out blocks out oof timeline & popular

* more routes

* Test blocked posts in threads

* Support blocks on record embeds

* notifs

* Update pds test snapshots w/ block state

* fix search

* Update pds actor search snapshots w/ blocks

* test

* tests

* update lex

* fix up from returning blocking as record

* ci for branch

* fix build issue in bsky

* more blocks tests

* fix actor search test

* hook getBlocks up to router

* bugfix & actually test lol

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-04-28 19:36:02 -05:00
Daniel Holmgren 63bef856cc Return on subscription abort (#921)
return on subscription abort
2023-04-27 23:34:03 -05:00
devin ivy 6b3d5c133e Misc notification and suggestion optimizations (#920)
* Background notification creation/deletion

* Add todo

* Update tests for backgrounded notifs

* Add index to notifications supporting record deletion

* Use profile_agg to power getSuggestions

* Fetch record blocks separately from getting notification resultset

* Update background queue to drain pending tasks
2023-04-27 19:10:21 -04:00
devin ivy da6ee3908a Background notification creation and deletion (#919)
* Background notification creation/deletion

* Add todo

* Update tests for backgrounded notifs

* Undo backgrounding of notification creation

* Revert "Undo backgrounding of notification creation"

This reverts commit 507353a9802da75a232fc245ede30d47dab74e5f.
2023-04-27 17:12:18 -04:00
devin ivy 5249e52f36 Lengthen pass reset and invite codes (#912)
* Lengthen pass reset and invite codes, invalidation on reset

* Tidy

* Adjust token format for reset and deletion

* Update invite codes to use same token gen as pass reset and acct deletion
2023-04-27 14:26:31 -04:00
devin ivy 7d1243f53a Limit concurrency of pds background tasks (#917) 2023-04-27 14:07:46 -04:00
Daniel Holmgren 384e739a3b Allow admins to update handles to reserved handles (#916)
* allow admins to update handles to reserved handles

* tidy
2023-04-27 12:02:47 -05:00