Commit Graph

2872 Commits

Author SHA1 Message Date
Devin Ivy 92995859dc Fix bsky getPosts 2023-04-24 09:03:39 -04:00
Daniel Holmgren 553f59923c Get posts method (#876)
* implement getPosts method

* tests

* bsky tests

* comment out test

* Handle stripping record-with-media viewer info in bsky tests

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-04-23 23:43:37 -05:00
devin ivy 47890d65b9 Add labeler functionality to the appview (#851)
* Add moderation and labeling model to bsky appview

* Carry over moderation service from pds

* Pass over bsky moderation service to get it working, missing blob support

* Carry over label service from pds to bsky

* Reorg bsky http auth, implement admin auth

* Carry over relevant admin endpoints pds to bsky

* Retrofit bsky admin routes from pds impls

* Implement resolve handle on appview

* Avoid loop in bsky handle resolution

* Add bsky appview to dev-env, opt-in

* Fix bsky searchRepos with empty term, tidy

* Include blobs on bsky admin views

* Stop resolving taken-down blobs on appview

* Tidy

* Carry over pds moderation tests to bsky

* Support image cache invalidation on bsky appview

* Add missing changes for bsky tests

* Test takedowns in bsky views

* Test takedowns on bsky notifs

* Separate blob resolution logic into standalone helper

* Carry over labeler from pds to bsky

* Update labelers for bsky usage

* Configure labeler on appview, tidy blob-resolver

* Add labels to views for posts and actors

* Update snapshots on bsky appview to include labels

* Carry over labeler tests from pds to bsky

* Tidy-up labeler tests for bsky appview

* Update bsky seeds and snapshots for keyword labeling

* Tidy

* In bsky appview ensure label.neg is modeled as a boolean, not an integer bit. Add select column when checking blob takedown.

* Include labels on appview listNotifications, fix blob resolver content-type, tidy
2023-04-23 23:58:10 -04:00
devin ivy 05e6ebed63 Bsky appview admin and moderation endpoints (#840)
* Add moderation and labeling model to bsky appview

* Carry over moderation service from pds

* Pass over bsky moderation service to get it working, missing blob support

* Carry over label service from pds to bsky

* Reorg bsky http auth, implement admin auth

* Carry over relevant admin endpoints pds to bsky

* Retrofit bsky admin routes from pds impls

* Implement resolve handle on appview

* Avoid loop in bsky handle resolution

* Add bsky appview to dev-env, opt-in

* Fix bsky searchRepos with empty term, tidy

* Include blobs on bsky admin views

* Stop resolving taken-down blobs on appview

* Tidy

* Carry over pds moderation tests to bsky

* Support image cache invalidation on bsky appview

* Add missing changes for bsky tests

* Test takedowns in bsky views

* Test takedowns on bsky notifs

* In bsky appview ensure label.neg is modeled as a boolean, not an integer bit. Add select column when checking blob takedown.
2023-04-23 23:43:08 -04:00
Daniel Holmgren f30887bc1c Clean up Did-resolver (#842)
* refactor did-resolver

* clean up deps

* some fixups + caching utilities

* couple helpers around cache invalidation

* fix expired check

* change cache semantics

* use map instead of record

* stale + expired

* tests

* clear entry method

* clear missing dids

* add missing type
2023-04-23 13:36:01 -05:00
bnewbold 6ebf810a48 minor Makefile tweak: run-dev-pds, not run-pds (#875)
Makefile: run-dev-pds, not run-pds
2023-04-23 12:31:57 -05:00
Daniel Holmgren 01578bc5eb Better handle races in createAccount (#868)
* better handle races in create account

* better locking

* cleanup

* tidy comment
2023-04-23 12:31:25 -05:00
Daniel Holmgren 2b5347b534 Remove profile labels (#866)
* moderation tweks

* fix: typo. (#818)

* Create invite codes for many accounts (#825)

* create invite codes for many accounts

* test

* Revamp dev env (#796)

* fix up a couple of tsc errors in app view merge

* wip

* use dev-env for appview tests

* process all in blob resolver

* another test fix

* fixed missed merge conflict

* fix one more merge conflict

* fix popular test

* Sync test env with labeler changes (#836)

fix test-env

* Remove extraneous info on getRecord (#835)

* remove extraneous info on getRecord

* fix test-env

* Fix duplication of constants in the crypto package (#819)

* App passwords (#826)

* app password lex & auth chnages

* scrypt things

* implemented app password refresh tokens

* db tidy & migration

* revocation + bugfixin

* tests, listing passwords & cleanup

* Update packages/pds/src/db/scrypt.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Update packages/pds/src/db/scrypt.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* pr feedback

---------

Co-authored-by: devin ivy <devinivy@gmail.com>

* remove profile labels

* rm temp branch from workflow

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: S. Ota <1632335+susumuota@users.noreply.github.com>
Co-authored-by: Ilya Siamionau <ilya@marshal.dev>
2023-04-23 12:18:24 -05:00
devin ivy d86cf03cd8 More forgiving config for empty db schema (#871)
* Add more config for pds db pool

* More forgiving handling of empty db schema config
2023-04-22 14:26:31 -04:00
devin ivy 92322af49a Add more config for pds db pool (#867) 2023-04-21 22:13:41 -04:00
devin ivy a832c54de7 PDS moderator credentials (#863)
* Setup config and auth verifiers for moderators

* Enforce admin vs. moderator access on PDS admin/server endpoints

* Tidy
2023-04-21 20:58:34 -04:00
Paul Frazee 5b4f506150 Bump @atproto/api@0.2.8 (#865) 2023-04-21 19:12:28 -05:00
Daniel Holmgren 89109ef32d Format commits outside of tx (#860)
* format commits outside of tx

* fix tests

* fix bsky test

* use persistent cache

* reuse repo storage between attempts to get caching benefits

* better error handling

* tests

* rm failing sqlite test

* pr feedback
2023-04-21 18:13:04 -05:00
Daniel Holmgren 4f1ad109af Increase getPopular likes to 8 (#862)
* increase get popular likes to 8

* use new like counts agg
2023-04-21 18:12:44 -05:00
Paul Frazee 5b0c2d7dd5 Rename CONTRIBUTERS.md to CONTRIBUTORS.md 2023-04-21 11:26:03 -05:00
devin ivy a0f0301241 Serve materialized post and profile aggregations (#859)
* Add migration for post & profile aggs to pds

* Post/profile agg db tweaks

* Post/profile aggregation queries on pds

* Add facility for performing work on db commit in pds

* Add background queue to pds

* Perform aggregations during indexing on background queue

* Tidy pds background queue, incorporate into labeler

* Fix pds onCommit, remove unused indexing dep, fix zeros for aggs

* Finish background tasks for view tests in pds

* Update aggregations on account deletion, tidy

* Update profile and post views to serve materialized aggregations
2023-04-21 12:01:23 -04:00
Paul Frazee 7679b2ea71 Create CONTRIBUTERS.md (#861)
* Create CONTRIBUTERS.md

* Update CONTRIBUTERS.md
2023-04-21 10:54:34 -05:00
devin ivy 587b24352d Materialize post and profile aggregations on write (#858)
* Add migration for post & profile aggs to pds

* Post/profile agg db tweaks

* Post/profile aggregation queries on pds

* Add facility for performing work on db commit in pds

* Add background queue to pds

* Perform aggregations during indexing on background queue

* Tidy pds background queue, incorporate into labeler

* Fix pds onCommit, remove unused indexing dep, fix zeros for aggs

* Finish background tasks for view tests in pds

* Update aggregations on account deletion, tidy
2023-04-21 11:52:45 -04:00
Alice a5016eba20 Fix README for api (#850)
=>
2023-04-20 12:57:46 -05:00
Daniel Holmgren 24f7183896 Improve streaming sync routes (#848)
* improve streaming sync routes

* pr feedback
2023-04-20 10:26:55 -05:00
Daniel Holmgren 9a8d7529a3 Skip locked repos instead of waiting for lock (#847)
* skip locked repos instead of waiting for lock

* some tests

* rm racing tests on profile

* subscribe repos test
2023-04-19 21:57:06 -05:00
Daniel Holmgren 2c24cf6c15 Bump node-postgres version (#846)
* Bump node-postgres version

* fix package.json things
2023-04-19 19:06:56 -05:00
Daniel Holmgren c71e351c38 Do not give notifs for a deleted subject (#843)
generally do not give notifs for a deleted subject
2023-04-19 16:24:18 -05:00
Ryan Barrett 4ecd3878a0 fix minor typos in getRepo lexicon (#838) 2023-04-19 10:26:46 -05:00
devin ivy c70cb2b6e5 Notifications from the appview (#829)
* Update notif lexicons for stateless seenAt param

* In-progress work on appview notifs

* Fix-up bsky notification methods

* Add appview notification table table

* Process notifications in bsky appview, test notif indexing

* Test bsky appview notification methods

* Tidy bsky notification tests

* Explicitly don't support seenAt notif params on pds

* Tidy bsky notifs tests

* Sync bsky notif handling with pds

* Remove stale comment

* Switch bsky notifs tests to use testenv
2023-04-18 17:55:03 -04:00
Daniel Holmgren 6446e8d1d6 App passwords (#826)
* app password lex & auth chnages

* scrypt things

* implemented app password refresh tokens

* db tidy & migration

* revocation + bugfixin

* tests, listing passwords & cleanup

* Update packages/pds/src/db/scrypt.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Update packages/pds/src/db/scrypt.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* pr feedback

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-04-18 13:47:04 -05:00
Ilya Siamionau 7d95e7573d Fix duplication of constants in the crypto package (#819) 2023-04-18 12:35:44 -05:00
Daniel Holmgren 6e2a3d3924 Remove extraneous info on getRecord (#835)
* remove extraneous info on getRecord

* fix test-env
2023-04-18 12:29:02 -05:00
Daniel Holmgren a687aca494 Sync test env with labeler changes (#836)
fix test-env
2023-04-18 12:28:40 -05:00
Daniel Holmgren d4963e5003 Revamp dev env (#796)
* fix up a couple of tsc errors in app view merge

* wip

* use dev-env for appview tests

* process all in blob resolver

* another test fix

* fixed missed merge conflict

* fix one more merge conflict

* fix popular test
2023-04-18 11:07:41 -05:00
Daniel Holmgren 9e1bb49957 Create invite codes for many accounts (#825)
* create invite codes for many accounts

* test
2023-04-18 10:44:34 -05:00
S. Ota 2360ffea62 fix: typo. (#818) 2023-04-17 11:38:18 -07:00
devin ivy 6a39ebf9f7 Stop sequencer listener in outbox when subscription aborts (#828)
* Update types to allow for abort signal in xrpc subscriptions

* Support abort signal in xrpc-server subscriptions

* Stop sequencer listener in outbox when subscription aborts
2023-04-17 14:22:03 -04:00
Daniel Holmgren d8b50c73e4 Add admin.updateAccountEamil (#812)
* -add admin capability to update account email

* pr feedback
2023-04-13 14:43:53 -05:00
devin ivy 4aa9103fb5 Minor query optimization for getBlocksForCommits() (#811) 2023-04-13 14:39:49 -05:00
Daniel Holmgren efb1cac2bf Label reversal fix (#809)
* Reproduce issue with empty negateLabelVals

* normalize str arrays

* Reproduce issue with reversing labels on repo

* Use multiple labels for record labeling test

* join with space

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-04-12 23:48:47 -05:00
Daniel Holmgren aa46ad1e1c Admin labeling (#801)
* adding to moderation flows

* some bugfixing in labels

* hardcode profile nsid

* make labels off moderation action

* db updates

* wip

* report labels in admin views & reverse moderation actions

* Test admin get record and repo w/ labels

* update db

* fix

* exclude negs from labels

* exclude neg on moderation views as well

* Check-in missing lex

* Check-in missing lex

* In-progress admin label tests

* Test label creation/reversal via actions

* Admin label test snapshots (#808)

* new snapshots for label on user

* fix get moderation action snap

* fix dev-env

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-04-12 22:53:32 -05:00
bnewbold 6592fcd6eb Lex: additional report types (#788)
* lex: more report types

* lex: basic definitions of report reasonTypes

* package/api codegen: report types

* packages/pds codegen: report types

* packages/bsky codegen: report types; also roll-up many other changes
2023-04-12 22:32:57 -05:00
Paul Frazee 55e90ac865 @atproto/api@0.2.7 2023-04-12 20:16:42 -07:00
Devin Ivy 3f632d8bdb v0.1.4 2023-04-12 18:09:48 -07:00
Devin Ivy 09d1df680e v0.2.6 2023-04-12 18:08:18 -07:00
Daniel Holmgren 46e7ba986d Update one label (#807) 2023-04-12 20:02:51 -05:00
Daniel Holmgren f6aef2b50f Fix race condition in labeling (#806)
* Fix race condition in labeling

* comment
2023-04-12 19:39:12 -05:00
Daniel Holmgren d6ac47d76e Label profiles in feeds (#805)
* Label profiles in feeds

* send empty array
2023-04-12 18:14:23 -05:00
Daniel Holmgren 521c1c1edb Admin update handle (#803)
* admin route for updating accoutn handle on services domain

* tests

* Update lexicons/com/atproto/admin/updateAccountHandle.json

Co-authored-by: devin ivy <devinivy@gmail.com>

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-04-12 17:23:58 -05:00
Daniel Holmgren 0762e8b7f6 Label bugfixes (#802)
* some bugfixing in labels

* hardcode profile nsid
2023-04-12 17:23:32 -05:00
dholms 30ee2c3dc9 Remove other branches from workflows 2023-04-12 13:03:09 -07:00
Daniel Holmgren d370ba41a7 Reserved handles (#800)
* more reserved handles

* workflow to build this branch
2023-04-12 15:01:55 -05:00
Daniel Holmgren c41232b2ef Fix DB schema in rebase tests (#799)
fix typoed db schema in test
2023-04-12 14:34:39 -05:00
Daniel Holmgren 8b1da9fed8 Labeling (#789)
* take notifs off of message queue

* fix order of notif handling

* table

* reintroducing message queue

* setting up user

* added labeler classes

* tidy + hook up labeler to config / service start

* hooking up p-queue to labeler

* rip out message queue

* drop mq tables

* rm stream consumers

* keyword labeler

* tidy + some tests

* work labels into views

* update snaps

* labeled images in dev-env

* snaps

* labeler tests

* more labels

* Update packages/pds/src/api/app/bsky/notification/listNotifications.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Update packages/dev-env/src/mock/index.ts

* Update packages/dev-env/src/mock/index.ts

* Update packages/pds/src/labeler/util.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Update packages/pds/tests/labeler/hive.test.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* pr feedback + migration change

* fix lower case keywords issue

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-04-12 14:34:18 -05:00