1093 Commits

Author SHA1 Message Date
Patryk
d661a60357
Add lexicon LexXrpcParameters, Object required properties check (#1040)
* Add lexicon LexXrpcParameters, Object required properties check

Now schemas like the one below won't validate
```
...
{

  "type": "object",
  "required": ["foo"],
  "properties": {}
}
...
````

* Fix lexicons with missing required fields

* Improve lexicon user type error messages and document why it needs to be `z.custom`
2023-05-23 01:02:36 -04:00
Daniel Holmgren
8059e07d8a
Custom feeds (#1001)
* actor type in profiles

* skeleton lex

* more schemas

* first pass at feed composition

* cleanup services

* more tidying

* fix up tests

* bookmarking schemas

* make feeds records instead of actors

* some cleanup

* fleshing out feeds as records

* wip

* working into cfg & dev-env

* fix appview test util

* bookmarking + views

* tidy

* add displayName

* db migration

* wip test

* blob processing

* first test working!

* Sync-up snapshots

* wip

* Fix pds ctx type

* Sync lexicons on pds and api

* wip

* fully refactor

* Prep for open union in reply data in bsky appview, sync-up snapshots

* msgs & ports

* Make pds did cache sqlite-friendly

* Start scaffolding more feedgen tests

* revamp appview tests

* revamp pds tests

* merge in dev env utils

* ensure plc close

* add feed gen to dev-env

* second feed gen

* named feeds in mock

* fix issue with dev-env pds appview

* update bsky snaps

* Flesh-out some feed gen tests

* Fix feed gen reply test

* Test feed bookmarking

* update skeleton defs

* merge

* Sync-up pds with updated feed skeleton contract

* Fix feed bookmark viewer state

* add feed embeds

* Test inaccurate feed gen repost

* Tighten some error behaviors around feed gens

* generator view in record embed

* test

* bookmarked/subscribed -> saved

* updated snaps

* name & descript on alices feed

* add like counts

* proposed lex

* tweak

* fetch feed generator details

* add describeFeedGen to dev-env

* adds tests for getFeedGenerator

* Update lexicons/app/bsky/feed/getFeedGenerator.json

Co-authored-by: devin ivy <devinivy@gmail.com>

* update describe

* add cid to feed gen view

* add alice feed gen avatar

* generator displayName required

* update snaps

* Fix dev env public url (#1058)

* Fix the public url used by dev-env

* move publicUrl location

---------

Co-authored-by: dholms <dtholmgren@gmail.com>

* add defs for feed gen prefs & decrease displayName length

* build branch

* updated how saved feeds works

* cleanup lex

* mount route

* Starter bsky algos (#1046)

* implement some starter bsky algos

* add team dids

* tidy

* add quotepost check to whats hot classic

* tests

* carve out feedgen section of pds

* apply limits

* add describeFeedGenerator route

* Whats hot new (#1060)

* map out new whats hot

* only available in postgres

* Fix dev env public url (#1058)

* Fix the public url used by dev-env

* move publicUrl location

---------

Co-authored-by: dholms <dtholmgren@gmail.com>

* add defs for feed gen prefs & decrease displayName length

* build branch

* updated how saved feeds works

* build algos branch

* remove relative imports to @atproto/api

* update hot classic name

* rm build

---------

Co-authored-by: Paul Frazee <pfrazee@gmail.com>

* v0.3.6

* v0.3.7

* v0.1.10

* update name of skyline

* build branch

* aws build

* getFeedGenerators test

* For now only use new replyRef union for custom feeds, for compat

* Don't use replyRef union on new feeds yet

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
2023-05-19 18:31:51 -04:00
devin ivy
0899510e6f
Db migration to prep for custom feeds (#1074) 2023-05-19 16:40:47 -04:00
bnewbold
d8785649c6
auto-label: less sensitive for some classes (#1071) 2023-05-18 16:31:37 -07:00
Daniel Holmgren
cf36b363b4
Rkey len (#1066)
* set a lex limit on rkey length

* enforce in repo package

* rm checks in repo pkg

* lift constraint on update
2023-05-18 17:57:59 -05:00
devin ivy
e57c297a82
Handling deleted content on admin/moderation methods (#1045)
* Update lexicon for missing repo/record data in admin views

* Serve missing repo/record variant in admin views

* Test actioning and presenting unknown records/repos, fix on pds
2023-05-17 15:28:52 -04:00
bnewbold
2fc058046b
more labeling tweaks (#1053)
* labeling: include 'sex toy' as 'sexually suggestive'

* labeling: label 'hanging' as 'corpse' (for now)

* labeling: include 'underwear' as 'sexual'
2023-05-16 17:32:46 -07:00
Daniel Holmgren
6ad16b2fa4
Ensure sequencing is the last query in each tx (#1036)
* ensure that inserting repo_seq is the last thing to happen in every tx

* another
2023-05-15 14:09:27 -05:00
Daniel Holmgren
47d30c1796
Revamp dev env (#1031)
* wip

* wip

* fully refactor

* msgs & ports

* revamp appview tests

* revamp pds tests

* ensure plc close

* ensure pds appview is hooked up when not running appview

* move service setup to dev env

* move admin auth headers to dev-env

* fix service headers

* missing port

* fixing ports

* start services in order

* startup plc immediately

* rm dead code

* fix up build

* rm log
2023-05-15 12:28:36 -05:00
Daniel Holmgren
231729db72
Return email in mod views according to lexicon (#885)
* return email in mod views according to lexicon

* Update packages/pds/src/services/moderation/views.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* updated snaps

* tidy

* Fix snapshot

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-05-15 11:43:06 -04:00
Daniel Holmgren
e4ffcf0034
service <> service auth (#1026)
* wip

* working into cfg & dev-env

* fix appview test util

* bump xrpc server version

* no check did on deciding to mount appview proxy
2023-05-12 16:45:28 -05:00
devin ivy
1cbffd63ef
Reports by active action type (#1028)
* Lexicon updates for getting mod reports by active action type

* Update pds and bsky to support getting mod reports by active action type
2023-05-12 16:45:15 -05:00
Daniel Holmgren
5804716504
Bring your own DID (#1011)
* allow bringing your own did

* tests + tidy

* one more check/test

* fix typo

* better err names
2023-05-12 16:18:49 -05:00
dholms
2753a5a739 v0.1.8 2023-05-11 16:30:36 -05:00
dholms
7f72478e15 v0.1.7 2023-05-11 16:28:48 -05:00
Paul Frazee
df6ed7d5c0
Add account preferences APIs (#1013)
* Add lexicons for account preferences

* Move prefs to app.bsky, codegen

* Setup model and services for user prefs

* Setup xrpc methods for prefs

* Test preferences, fixes

* Tidy

* Tidy

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-05-11 15:44:00 -05:00
dholms
5fd5c869ea bump pds version 2023-05-11 15:01:01 -05:00
Daniel Holmgren
e32dc025e8
Mute lists (#986)
* possible schemas

* more lexicons

* codegen

* tables

* hook up processor

* getlist route

* add getListBlocks route

* migration

* didnt save something

* add reason + facets

* description facets

* get facets into db

* add to accnt deletion tests before i forget

* tests & bugfixes

* quickfix

* move defs to .graph

* add purpose

* remove unused actor defs

* add creator info to list view

* refactor to graph service

* lexicons

* impl

* writer methods

* bug in accnt service

* handle blobs

* tests

* fix typos

* tidy

* fix snap

* pr feedback

* missed drop index in migration

* remove listitem reason

* allow puts for lists

* make mute lists idempotent

* test mute list updates

* fix up

* fix err test

* update snap
2023-05-11 14:29:08 -05:00
Patryk
a7c2b4ae11
Add block record duplicate detection (#1027)
* Add block record duplicate detection

* Update packages/pds/src/services/record/index.ts

---------

Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>
2023-05-11 11:00:24 -04:00
bnewbold
7368020068
pds: place-holder robots.txt, and a text base-level route (/) (#995)
* pds: place-holder robots.txt and / (as txt)

robots.txt: want to be explicit about allowing crawling (generically)

/: a bit of dev-experience polish. in prod we already redirect this
route, but may be helpful for self-hosting folks and people just
exploring

* pds: move basic routes (health, robots.txt, /) to separate file
2023-05-10 17:08:52 -07:00
Daniel Holmgren
89b783eec9
Update how sequencer invalidation works (#1014)
* invalidatedBy -> invalidated

* fix test

* dont remake fkey constraint sqlite

* sqlite migration

* appeasing sqlite

* sqlite finally happy?

* Fix sqlite migration, run proxy tests thru postgres when configured

* Fix pg schemas in proxy tests

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-05-10 11:01:56 -04:00
devin ivy
977202fa2b
Fix indexed record datetimes (#1009)
* Normalize record datetimes used for sort to simplified iso

* Update tests and snapshots for alternate valid datetime formats

* Fix notifications race during record update
2023-05-09 12:53:33 -04:00
devin ivy
c3c75322bf
Sync-up account deletion indexes migration (#1006) 2023-05-09 12:52:56 -04:00
Daniel Holmgren
d30b0f93e1
Disable account invites (#1004)
* enable disabling account invites

* xtra test

* couple fixes

* allow re-enabling user accoutn invites

* add in getter route for if invites are disabled or not

* fix admin search test

* gneerated type on user account
2023-05-09 09:24:12 -05:00
Daniel Holmgren
51503384b6
No cache invalidation on blob deletion (#1008)
no invalidate on blob deletion
2023-05-09 08:57:45 -05:00
Daniel Holmgren
b20a5045a2
Rebase route (#948)
* tidy rebase query

* add rebase route

* bring tests up to speed

* add tx check
2023-05-09 05:54:48 -05:00
Daniel Holmgren
c8e6adbbf5
Clean up what's hot (#1002)
* changes to clean up whats hot

* fix test

* add quote posts back in & use underwear label instead of no-promote

* tidy
2023-05-08 18:26:11 -05:00
devin ivy
41793b7ff9
Add indexes to support account deletion (#1003)
* Add indexes to support account deletion

* Temporarily skip index creation in migration
2023-05-08 18:47:25 -04:00
Wes Todd
b01e47b617
fix: updated repository field for packages (#994) 2023-05-08 11:38:42 -04:00
Daniel Holmgren
aabbf43a7f
Blessed hell thread (#988)
give notifs in blessed hell thread
2023-05-05 20:51:57 -05:00
Daniel Holmgren
8e5f2bd2dc
Do not give deeply nested notifs (#987)
do not give deeply nested notifs
2023-05-05 19:00:02 -05:00
Daniel Holmgren
bfe59e2bd3
Do not filter on negative labels on getPopular (#985)
do not filter on negative labels on whats hot
2023-05-05 18:59:49 -05:00
bnewbold
8c19ce991a
moderation action 'escalate' type (#976)
* lex: 'escalate' mod action type

* api: codegen for escalate mod action type

* pds: codegen+impl esclate mod action type

* bsky: codegen+impl esclate mod action type

* pds: prettier:fix

* Tidy

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-05-04 12:32:27 -07:00
Daniel Holmgren
b75436b9cb
Delete user state on account delete (#970)
* Delete user state on account delete

* blocks too
2023-05-03 22:39:57 -05:00
Daniel Holmgren
015fe1226c
Delete dereferenced blobs (#959)
* delete dereferenced blobs

* invalidate cache for delete blobs

* start testing

* test & cleanup

* rm blob cleanup logic on rebases

* make disk blob deletes idempotent

* move blob deletion to background queue
2023-05-03 18:34:55 -05:00
Daniel Holmgren
b182344fef
Avoid sequencer retry loop (#958)
* avoid seq loop

* tidy

* comments
2023-05-02 17:07:18 -05:00
devin ivy
a1666776b3
Log requests to subscribeRepos (#957) 2023-05-02 17:37:23 -04:00
Daniel Holmgren
4555596212
Filter server-side labels (#956)
filter server side labels
2023-05-02 16:00:41 -05:00
devin ivy
b1493650f4
Perform account deletion async (#951)
* Perform takedown then background hard-deletion process for deleteAccount

* Remove transactions from account deletion process, perform steps sequentially
2023-05-02 14:24:59 -04:00
devin ivy
142c77cadd
Disallow writes of bad records (#955)
* Disallow writes of bad records

* Increase obj depth to hit stack issue on more systems
2023-05-02 13:44:07 -04:00
Daniel Holmgren
86fae264d3
Sequencer race condition (#906)
* wip

* split out seq invalidations

* wip

* fix

* comments

* 2 db hits

* cleanup

* re-add comment
2023-05-02 11:39:25 -05:00
bnewbold
5fd71c099b
appview: tweak repo error log line to not clobber 'service' (#891)
Somehow the output of this log line has been ending up in datadog under
the wrong 'service' (with the 'service' being an "wss://"-prefixed URL,
not "pds" or "bav" or whatever expected).

Tweaks the 'service' key, and some others that might have conflict.
2023-05-01 20:53:38 -05:00
Wes Bos
958c9ef6b3
add repository field to package.json (#895) 2023-05-01 20:43:32 -05:00
Daniel Holmgren
12c6d886eb
Deterministic DNS handle resolution (#883)
* enforce constraint for one handle per domain

* tweak + tests
2023-05-01 20:27:12 -05:00
Daniel Holmgren
1e24d3e820
Include nsfw flag for getPopular (#949)
add includeNsfw flag for whats hot
2023-05-01 15:50:21 -05:00
Daniel Holmgren
bff96544d1
Tweak whats hot (#945)
* tweak whats hot

* couple fixes & test
2023-05-01 14:10:48 -05:00
bnewbold
0b70f50eb4
pds: iterated Hive label parsing (#941)
Tune "porn", "sexual", and "nudity" label categorization, including
re-structuring of code paths.

This comes out of some image testing as well as review of Hive API docs.
2023-05-01 11:20:50 -07:00
devin ivy
9366734f6d
More aggressive backpressure on websocket writer (#929)
* Implement more aggressive backpressure on websocket writer, may be temporary

* Add heap snapshot signal to pds
2023-05-01 08:48:45 -04:00
Devin Ivy
3d1b3b3675 Add csp and nosniff when serving user uploads 2023-04-29 11:47:50 -04:00
Paul Frazee
01b5971907
api@0.2.11, pds@0.1.5 (#925)
* @atproto/api@0.2.11

* @atproto/pds@0.1.5
2023-04-28 19:41:01 -05:00