2489 Commits

Author SHA1 Message Date
Daniel Holmgren
d8b50c73e4
Add admin.updateAccountEamil (#812)
* -add admin capability to update account email

* pr feedback
2023-04-13 14:43:53 -05:00
devin ivy
4aa9103fb5
Minor query optimization for getBlocksForCommits() (#811) 2023-04-13 14:39:49 -05:00
Daniel Holmgren
efb1cac2bf
Label reversal fix (#809)
* Reproduce issue with empty negateLabelVals

* normalize str arrays

* Reproduce issue with reversing labels on repo

* Use multiple labels for record labeling test

* join with space

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-04-12 23:48:47 -05:00
Daniel Holmgren
aa46ad1e1c
Admin labeling (#801)
* adding to moderation flows

* some bugfixing in labels

* hardcode profile nsid

* make labels off moderation action

* db updates

* wip

* report labels in admin views & reverse moderation actions

* Test admin get record and repo w/ labels

* update db

* fix

* exclude negs from labels

* exclude neg on moderation views as well

* Check-in missing lex

* Check-in missing lex

* In-progress admin label tests

* Test label creation/reversal via actions

* Admin label test snapshots (#808)

* new snapshots for label on user

* fix get moderation action snap

* fix dev-env

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-04-12 22:53:32 -05:00
bnewbold
6592fcd6eb
Lex: additional report types (#788)
* lex: more report types

* lex: basic definitions of report reasonTypes

* package/api codegen: report types

* packages/pds codegen: report types

* packages/bsky codegen: report types; also roll-up many other changes
2023-04-12 22:32:57 -05:00
Paul Frazee
55e90ac865 @atproto/api@0.2.7 2023-04-12 20:16:42 -07:00
Devin Ivy
3f632d8bdb v0.1.4 2023-04-12 18:09:48 -07:00
Devin Ivy
09d1df680e v0.2.6 2023-04-12 18:08:18 -07:00
Daniel Holmgren
46e7ba986d
Update one label (#807) 2023-04-12 20:02:51 -05:00
Daniel Holmgren
f6aef2b50f
Fix race condition in labeling (#806)
* Fix race condition in labeling

* comment
2023-04-12 19:39:12 -05:00
Daniel Holmgren
d6ac47d76e
Label profiles in feeds (#805)
* Label profiles in feeds

* send empty array
2023-04-12 18:14:23 -05:00
Daniel Holmgren
521c1c1edb
Admin update handle (#803)
* admin route for updating accoutn handle on services domain

* tests

* Update lexicons/com/atproto/admin/updateAccountHandle.json

Co-authored-by: devin ivy <devinivy@gmail.com>

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-04-12 17:23:58 -05:00
Daniel Holmgren
0762e8b7f6
Label bugfixes (#802)
* some bugfixing in labels

* hardcode profile nsid
2023-04-12 17:23:32 -05:00
dholms
30ee2c3dc9 Remove other branches from workflows 2023-04-12 13:03:09 -07:00
Daniel Holmgren
d370ba41a7
Reserved handles (#800)
* more reserved handles

* workflow to build this branch
2023-04-12 15:01:55 -05:00
Daniel Holmgren
c41232b2ef
Fix DB schema in rebase tests (#799)
fix typoed db schema in test
2023-04-12 14:34:39 -05:00
Daniel Holmgren
8b1da9fed8
Labeling (#789)
* take notifs off of message queue

* fix order of notif handling

* table

* reintroducing message queue

* setting up user

* added labeler classes

* tidy + hook up labeler to config / service start

* hooking up p-queue to labeler

* rip out message queue

* drop mq tables

* rm stream consumers

* keyword labeler

* tidy + some tests

* work labels into views

* update snaps

* labeled images in dev-env

* snaps

* labeler tests

* more labels

* Update packages/pds/src/api/app/bsky/notification/listNotifications.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Update packages/dev-env/src/mock/index.ts

* Update packages/dev-env/src/mock/index.ts

* Update packages/pds/src/labeler/util.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Update packages/pds/tests/labeler/hive.test.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* pr feedback + migration change

* fix lower case keywords issue

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-04-12 14:34:18 -05:00
Daniel Holmgren
d8b975d8dd
Normalize handles on resolve (#795)
* normalize handles on resolve

* tidy & test
2023-04-12 10:49:49 -07:00
Daniel Holmgren
42e9c69b0a
Fix bug in getPopular (#794)
fix bug in getPopular
2023-04-12 12:24:15 -05:00
Daniel Holmgren
286c6bbef5
Remove mutes & takedowns from getPopular (#793)
remove mutes & takedowns from getPopular
2023-04-11 22:58:06 -07:00
Daniel Holmgren
a9e725bb19
Adjustable db pool size (#792)
* add pool size env

* Update packages/pds/service/index.js

Co-authored-by: devin ivy <devinivy@gmail.com>

* part int

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-04-11 22:01:10 -07:00
devin ivy
153542ae81
Drop PDS event queue tables (#790)
* take notifs off of message queue

* fix order of notif handling

* rip out message queue

* drop mq tables

* rm stream consumers

* Tidy

* Temporarily remove drop message queue migration

* Revert "Temporarily remove drop message queue migration"

This reverts commit 95c02e128a650469543f5b14418c4ff32c088d20.

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2023-04-11 21:56:01 -07:00
Daniel Holmgren
95ac1808ac
No notifs for any records from muted accounts (#791) 2023-04-11 21:55:20 -07:00
Daniel Holmgren
15762260fd
Sync/listRepos (#784)
* add getHostedRepos lex & route

* tests

* tidy

* rename
2023-04-11 17:16:13 -07:00
Daniel Holmgren
656be937a5
Lexicon: remove float support (#753)
remove float support for lexicon
2023-04-11 17:11:21 -07:00
Daniel Holmgren
fc4a34caf2
Fully deprecate handle on createSession (#761)
* fully deprecate handle on createSession

* fix tests
2023-04-11 17:10:35 -07:00
Daniel Holmgren
8d0cbb4c86
Create bulk invite codes (#783)
add bulk create invite code route
2023-04-11 17:09:53 -07:00
Daniel Holmgren
8ecc232796
Synchronous notifs (#785)
* take notifs off of message queue

* fix order of notif handling

* rip out message queue

* drop mq tables

* rm stream consumers

* Tidy

* Temporarily remove drop message queue migration

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-04-11 17:05:39 -07:00
bnewbold
aa20d20efb
More label lexicons (#672)
* com.atproto.label.label lexicon

For persisting un-signed label objects in repositories.

There will likely be a handful of XRPC query endpoints in the
com.atproto.label namespace in the near future.

* lexicons: com.atproto.label.subscribeLabels

* lexicons: com.atproto.label.query

* lexicons: label updates post lex-refactor

* lexicons: rename label.query to label.queryLabels

* lexicons: queryLabels larger batch sizes; improve description

* lexicons: add labels to profiles, posts, notifications

* lexicons: add negation ('neg') flag to labels

* lexicon: label vals on moderation actions

This is a mechanism to create (or negate) labels as part of a moderation
action. The fields are optional, so this is backwards compatible, and if
we wanted to embed entire labels in the future (instead of just "val"
strings), that would be possible using a different field name.

* lexicons: update label.label to label.defs#label

* codegen

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2023-04-11 12:44:19 -05:00
Devin Ivy
9697eaab89 Temporary, bump keepalive timeout on pds 2023-04-09 19:37:28 -07:00
devin ivy
6b55a95c88
Build timeline and author feeds from feed items index (#774)
* Index posts and reposts into feed_item table for building feeds

* Use feed_item table for building timeline and author feeds

* Apply feed item indexing to bsky app view

* Fix bsky appview tests, test getPopular

* Use feed item index to build feeds in bsky app view
2023-04-09 13:21:48 -07:00
devin ivy
2b8e1ac387
Index posts and reposts as feed items (#773)
* Index posts and reposts into feed_item table for building feeds

* Apply feed item indexing to bsky app view
2023-04-09 12:13:18 -07:00
devin ivy
fb65d97d29
Disable output validation when not in debug mode (#778)
* Disable output validation unless in debug mode

* Disable output validation unless in debug mode for bsky, tidy
2023-04-08 18:29:11 -04:00
Daniel Holmgren
8c71d0fe37
Cleanup appview merge (#772)
fix up a couple of tsc errors in app view merge
2023-04-07 10:44:00 -05:00
Daniel Holmgren
31d5258922
Fix pg sequencer test (#777)
* fix pg sequencer test

* tidy
2023-04-07 10:43:19 -05:00
devin ivy
2aeda0fbe9
Account for possible path prefix in cloudfront invalidation (#758) 2023-04-07 11:34:49 -04:00
Paul Frazee
f818d1c55e
Bump @atproto/api@0.2.5 (#776)
@atproto/api@0.2.5
2023-04-07 10:08:20 -05:00
Paul Frazee
5253c94417
Fix AtUri parse on hostnames that start with numbers (#775)
* Fix AtUri parse on hostnames that start with numbers

* @atproto/uri@0.0.2
2023-04-07 09:53:14 -05:00
devin ivy
4e83748545
Bsky app view (#716)
* Init pulling bsky app view from pds package into its own package, remove sqlite db dialect

* Cull bsky config, services, auth, etc.

* Sweep app view xrpc methods, tidy deps, add storage back for img server

* Run repo subscription on bsky app view

* Collapse db migrations down for bsky app view

* Tidy app view bin

* Remove mute functionality from app view, delegate to pds

* Initial tidy/culling of bsky app view tests

* Passing bsky app view db, server, and repo subscription tests

* Passing bsky app view duplicate-records tests

* Bsky app view test tidy/cull

* In bsky app view replace repo_root, ipld_block, did_handle with actor and record tables. Remove assertions/confirmations.

* Update bsky impl for simpler actor and record tables, removed asserion/confirmations. Skip indexing unknown collections.

* Setup actor handles by did in bsky app view

* Passing indexing tests on bsky app view

* Passing image tests on bsky app view

* Fix bsky actor reindexing, support custom lock id for testing repo subs

* Sweep bsky view tests, misc tests, passing

* Tidy bsky deps

* Include did in resized image uris

* Update bsky image process server to use getBlob

* Update image server tests, misc fixes

* Implement bsky blob resolver

* Wire local image processing server to local blob resolver, test blob resolver

* Tidy

* Tidy

* Tidy

* Tidy app view init

* Fix handle resolution, tidy

* Add utils for partitioning indexing by did

* Update repo sub to parallelize work per repo

* Dep tidy

* Tidy bsky tests for updated repo sub destroy()

* Update thead indexing to handle out-of-order posts

* Sketch out strategy in bsky for handling too-big commits

* Set content-type on sync.getBlob

* Add logging for failed transmissions in bsky blob resolver

* Tidy

* Tidy bsky repo indexing and supporting repo interfaces

* Sort in app view based on combo of creation and indexing times

* Fix types

* Add retry utils to bsky

* Add retries to http requests made by bsky

* Test repo indexing

* Update bsky db/model for lex refactor

* Update bsky lexicons for lex refactor

* Update bsky actor service for lex refactor

* Update bsky feed service for lex refactor

* Update bsky indexing service for lex refactor

* Update bsky repo subscription for lex refactor

* Tidy bsky repo sub

* Add unspecced endpoints to bsky app view, update entrypoint

* Update bsky xrpc utils for lex refactor

* Update bsky xrpc methods for lex refactor

* Update bsky test seeds for lex refactor, tidy api entrypoint

* Update bsky non-view tests for lex refactor

* Update bsky likes view test for lex refactor, minor fix

* Update bsky author feed tests for lex refactor, minor test util fix

* Update bsky follow, profile, repost, search view tests for lex refactor

* Update bsky timeline view tests for lex refactor

* Replace bsky out-of-order thread indexing logic

* Update bsky thread view tests for lex refactor, general test tidying

* Handle rebases and too-big commits in repo subscription, tracking commit data cid

* Tidy

* Ensure did resolver reports "not found" only when positively not found

* Handle tombstones and handle updates in bsky

* Test indexing handle updates and did tombstones

* Support cors on bsky

* Allow app view to serve most routes unauthed

* Tests for bsky unauthed views

* Tidy bsky service entrypoint and dockerfile

* Remove unused storage interfaces from bsky

* Bsky entrypoint and dockerfile fixes, tidy

* Add workflow for bsky build to aws

* Use more standard db env variables, make migration creds optional

* Make bsky repo subscription optional

* Fix lex->json serialization in bsky

* Split bsky actor sync state into its own table

* Skip invalid records on indexing full repo, tidy

* Tidy

* Leader test timing

* Tidy/lint

* Fix bsky config overrides
2023-04-06 18:47:46 -04:00
Daniel Holmgren
63abe1f081
Improve getPopular (#769)
Speed up getPopular
2023-04-06 13:03:04 -04:00
devin ivy
e8e5a88414
Fix for available invite creation (#767) 2023-04-06 09:33:29 -04:00
Paul Frazee
831b7dade9
api@0.2.4, pds@0.1.3 (#765)
* @atproto/api@0.2.4

* @atproto/pds@0.1.3
2023-04-05 18:12:14 -05:00
Daniel Holmgren
5ae26d9318
Improve admin invite code views (#764)
allow filter on search & add tests
2023-04-05 18:09:39 -05:00
Daniel Holmgren
83828c52cd
User invite codes (#757)
* getUserInviteCodes lex

* small change

* implement user invite code creation/getting

* transactionally ensure we dont allow duplicate creates

* testing & fixes

* clean up & allow admin creation for a particular user

* fix dev-env

* user -> accnt & add admin disable codes route

* proposed admin inv schemas

* more admin routes for inv codes

* tests for invite admin views

* pr feedback

* refactor & return usedBy + more details on getAccountInviteCodes

* adding invite info into moderation views

* tests passing
2023-04-05 17:13:26 -04:00
Paul Frazee
47cb6402cd
Bump pds@0.1.2, api@0.2.3 (#762)
* @atproto/pds@0.1.2

* @atproto/api@0.2.3
2023-04-04 17:05:51 -05:00
Daniel Holmgren
ba527918e4
Paginate listRecords based on cursor (#751)
paginate listRecords based on cursor
2023-04-04 15:36:44 -05:00
Paul Frazee
25c23b6b61
Add email to createSession (#760)
Add email to createSession and update api to store email in the session object

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-04-04 16:06:19 -04:00
Daniel Holmgren
eb488b96f5
lex refactor hot fixes (#745)
* filter embeds from get popular

* Truncate profile info to satisfy validation (#746)

* Remove trailing replacement character when utf8-truncating (#747)

* Truncate profile info to satisfy validation

* Fix utf8 truncation w/ replacement character

* filter replies

* delete embeds from records on getPopular

* Update profile display name and description lengths to be based on graphemes (#748)

* @atproto api v0.2.1

* Tidy

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-04-04 15:56:53 -04:00
Paul Frazee
1014938520
Pkg bumps after #756 (#759)
* @atproto/0.2.2

* @atproto/pds@0.1.1
2023-04-04 14:33:23 -05:00
Daniel Holmgren
095652b75b
Include email on server.getSession (#756)
* include email on getSession

* fix up tests & return email on createSession

* fix more tests

* cleanup
2023-04-04 13:49:38 -05:00