315 Commits

Author SHA1 Message Date
Daniel Holmgren
396698f218
Filter out self & follows from suggestions (#498)
dont suggest self or follows
2023-02-02 16:48:41 -05:00
Daniel Holmgren
773f9e304d
Account deletion (#488)
* wip

* fleshing out repo storage

* fleshing out sql storage

* cleaning things up

* fix up tests

* dumb bug - commit log reversed

* rm staging in favor of commiting diffs to blockstore

* clean up benches

* fixing up sql storage

* some caching for sql repo store

* pr feedback

* migration

* wip

* migraiton test

* unclear param

* sql repo storage tests

* rm unused code

* fix up some diff code

* pr feedback

* enum for action types

* missed some

* wip

* ripping out auth lib

* more auth cleanup

* another lurker

* wip better sync primitives

* wip

* improving diffs & sync

* tests working!

* actually implemented checkout lol

* simplify interface & improve error handling

* writing sql storage code

* fixing up tests

* testing & bugfixes

* checkouts return records instead of cids

* one last refactor lol

* missed one

* handle other cid codecs on incoming car verification

* tests + tricky bugs

* unneeded blockstore method

* trim mst on del instead of save

* cleanup comment

* dont resolve did for every commit

* use "commit" instead of "root"

* getRoot -> getHead

* pr feedback

* very silly bug fix

* improve sync output

* reorging + sync of particular records

* serve & verify proofs. also rename some ipld methods

* fix up sync issue in mst

* find reachable records form carfile

* getRecord xrpc method

* pr feedback

* better migration test

* check migraiton result

* fixing up a couple things for pg

* explicit migrateTo

* async exceptions

* ipld car mimetype + remove updateRepo

* Update module publish scripts (#478)

* Update pds package publishing scripts

* Update auth package publishing scripts

* Update crypto package publishing scripts

* Update did-resolver package publishing scripts

* Update handle package publishing scripts

* Update xrpc-server package publishing scripts

* Update common package publishing scripts

* Update plc package publishing scripts

* Update uri package publishing scripts

* Update repo package publishing scripts

* Sort "suggested follows" by number of posts (#477)

* return suggestions by post count

* pr feedback

* fix up PG pagination issue

* partiion commit-history & commit-blocks by user did

* some lexicons

reworking routes

request deletion flows

delete actor rows

migration for user-partitioned-cids

move creator to be on ipld_block

migration tests

* delete records & repos

* delete blobs

* hook it up in route

* pettier ignore email templates

* testing & bugfixes

* testing blobs & bugfixes

* pr feedback

* make deletion test more robust

* change out handle for did on account deletion

* small cleanup

---------

Co-authored-by: Paul Frazee <pfrazee@gmail.com>
2023-02-02 12:34:00 -06:00
Daniel Holmgren
05855fcff3
Loosening image constraints (#503)
* loosening img constraints

* fixing up tests
2023-01-30 18:59:19 -05:00
Daniel Holmgren
75e14ae102
Feature branch: storage & sync revamp (#446)
* wip

* fleshing out repo storage

* fleshing out sql storage

* cleaning things up

* fix up tests

* dumb bug - commit log reversed

* rm staging in favor of commiting diffs to blockstore

* clean up benches

* fixing up sql storage

* some caching for sql repo store

* pr feedback

* migration

* wip

* migraiton test

* unclear param

* sql repo storage tests

* rm unused code

* fix up some diff code

* pr feedback

* enum for action types

* missed some

* wip

* ripping out auth lib

* more auth cleanup

* another lurker

* wip better sync primitives

* wip

* improving diffs & sync

* tests working!

* actually implemented checkout lol

* simplify interface & improve error handling

* writing sql storage code

* fixing up tests

* testing & bugfixes

* checkouts return records instead of cids

* one last refactor lol

* missed one

* handle other cid codecs on incoming car verification

* tests + tricky bugs

* unneeded blockstore method

* trim mst on del instead of save

* cleanup comment

* dont resolve did for every commit

* use "commit" instead of "root"

* getRoot -> getHead

* pr feedback

* very silly bug fix

* improve sync output

* reorging + sync of particular records

* serve & verify proofs. also rename some ipld methods

* fix up sync issue in mst

* find reachable records form carfile

* getRecord xrpc method

* pr feedback

* better migration test

* check migraiton result

* fixing up a couple things for pg

* explicit migrateTo

* async exceptions

* ipld car mimetype + remove updateRepo

* Update module publish scripts (#478)

* Update pds package publishing scripts

* Update auth package publishing scripts

* Update crypto package publishing scripts

* Update did-resolver package publishing scripts

* Update handle package publishing scripts

* Update xrpc-server package publishing scripts

* Update common package publishing scripts

* Update plc package publishing scripts

* Update uri package publishing scripts

* Update repo package publishing scripts

* Sort "suggested follows" by number of posts (#477)

* return suggestions by post count

* pr feedback

* fix up PG pagination issue

* partiion commit-history & commit-blocks by user did

* fix migration type

* switch pk indexes

Co-authored-by: Paul Frazee <pfrazee@gmail.com>
2023-01-26 18:09:55 -06:00
Daniel Holmgren
6f780bf670
Remove scenes (#483)
* wip

* wip

* remove getAssertions & fix up notifs test

* remove did param on updateProfile
2023-01-26 15:38:45 -06:00
Paul Frazee
49d31f0a2c
Add missing 'avatar' field in app.bsky.graph.getFollows (#494) 2023-01-26 14:14:00 -06:00
devin ivy
9834c2f5ef
Allow login using email (#492)
* Allow login using email, deprecate handle input in session.create

* Remove handle input from session.create lexicon
2023-01-26 13:08:56 -05:00
devin ivy
13dd58bb15
Feature: admin moderation views (#472)
* Model mod actions for actor takedowns

* Add administration lex schemas for moderation actions

* Initial implementation of takedown mod action and reversal

* Refactor admin actions into a service

* Check for account takedown when creating new session or refreshing session

* Sweep usage of getUser() and getUserByEmail() for takedowns

* Begin sweep for account takedowns

* Tidy takedown checks

* Handle post author takedowns

* Fix tests and migrations for mod takedown

* Fix xrpc-server delayed read of blob payloads

* Test takedown mod action, misc fixes

* Remodel lex schemas for admin moderation actions

* Remove delcaration cid from actor takedowns, lex schema tidy

* Add notes

* Move moderation methods to com.atproto, based around repo rather than actor

* Moderation reports of repos and records

* Implement moderation report resolution method

* Tidy

* Place moderation-init migration after mute-init

* Allow services to reference each other

* Model record takedowns in db

* Lex schemas for record takedowns

* Service/methods for record takedowns, fix report view subject

* Apply record takedown to posts in feeds

* Test record takedown effects, fix reversal

* Support server moderation flag and acknowledge actions, fix action/report resolution for records

* Track repo takedowns on repo_root rather than did_handle

* Move repo/record reporting under com.atproto.report

* Colocate all moderation functionality into moderation service, combine from admin and repo services

* Tidy

* Tidy

* Unify mod report and action methods with new repo/record ref schemas

* Schemas for admin moderation views

* Tidy/codegen

* Fix test snapshot

* Break pds admin methods into separate files

* Implment admin repo search method

* Reorganize existing moderation views into the ModerationViews

* Lex schemas for admin get repo/record, implement get repo

* Implement admin get record view

* Tidy/codegen

* Tidy, fix snapshot

* Tidy, swap filenames

* Implement admin get moderation action view

* Implement admin get moderation report view

* Implement and test admin views for mod actions and reports

* Dry-up some admin and moderation-related seeds in the tests

* Fix xrpc fetch usage for newly required duplex option

* Admin view of repo and record details (#463)

* Reorganize existing moderation views into the ModerationViews

* Lex schemas for admin get repo/record, implement get repo

* Implement admin get record view

* Tidy/codegen

* Tidy, fix snapshot
2023-01-18 18:01:04 -05:00
devin ivy
ef487900f6
Server moderation action/resolution and reporting (#458)
* Model mod actions for actor takedowns

* Add administration lex schemas for moderation actions

* Initial implementation of takedown mod action and reversal

* Refactor admin actions into a service

* Check for account takedown when creating new session or refreshing session

* Sweep usage of getUser() and getUserByEmail() for takedowns

* Begin sweep for account takedowns

* Tidy takedown checks

* Handle post author takedowns

* Fix tests and migrations for mod takedown

* Fix xrpc-server delayed read of blob payloads

* Test takedown mod action, misc fixes

* Remodel lex schemas for admin moderation actions

* Remove delcaration cid from actor takedowns, lex schema tidy

* Add notes

* Move moderation methods to com.atproto, based around repo rather than actor

* Moderation reports of repos and records

* Implement moderation report resolution method

* Tidy

* Place moderation-init migration after mute-init

* Allow services to reference each other

* Model record takedowns in db

* Lex schemas for record takedowns

* Service/methods for record takedowns, fix report view subject

* Apply record takedown to posts in feeds

* Test record takedown effects, fix reversal

* Support server moderation flag and acknowledge actions, fix action/report resolution for records

* Track repo takedowns on repo_root rather than did_handle

* Move repo/record reporting under com.atproto.report

* Colocate all moderation functionality into moderation service, combine from admin and repo services

* Tidy

* Tidy

* Unify mod report and action methods with new repo/record ref schemas
2023-01-06 12:59:47 -05:00
devin ivy
e9f5f23edc
Fix lexicon validation, do not allow undefined when required (#457)
* Fix lexicon validation, do not allow undefined when required

* Make term optional for actor search/typeahead
2023-01-05 12:04:31 -05:00
devin ivy
149da8218e
Remove unmapped post.viewer.muted property (#449)
Remove unused/unmapped post.viewer.muted property
2023-01-02 15:42:04 -05:00
devin ivy
aa626c3dc4
User muting (#445)
* Procedures and view for user muting

* Update profile and feed/thread methods for muting, update tests

* Tidy

* Test muting for timeline, actor feed, thread, profile views, misc fixes

* Test mute views, misc fixes

* Add missing file

* Add per-user notification index

* Omit mention and reply notifications from muted users

* Test timing leeway
2022-12-30 14:27:48 -05:00
Paul Frazee
1ede96c4ad
Improve image quality (#442)
* Increase the allowed dimenions of all images

* Increase the size and quality of served images

* Update tests for larger res
2022-12-26 15:51:54 -06:00
Daniel Holmgren
30ab0d341b
Refactor feeds & post threads (#423)
* factor out feed items & mystate

* propose feed items rewordk

* pr feedback

* revamping getTimeline

* feed service & cleanup

* bugfixin

* update timeline snapshots

* update getAuthFeed snapshot

* bugfixin

* revamp getPostThread

* fix up sync & votes tests

* bug in post thread

* fix replyCount

* fix ordering

* move uriImgBuilder to createServices
2022-12-20 20:38:37 -06:00
Dean van Dugteren
5be13ac3e3
fix: missing comma invalidates post.json (#433) 2022-12-20 17:59:55 -05:00
Ryan Barrett
4eb1b64665
app.bsky.feed.post lexicon: add textSlice description (#429)
say explicitly that start is inclusive, end is exclusive.
2022-12-19 13:28:01 -06:00
Paul Frazee
f5302ce139
Fixes to post media (#412)
* Fix typo

* Add failing test for attaching image to a post

* Tidy failing test

* Fix

* Expand the available image resize fit modes

* Add proper validation of glob-prefixed mimes (eg image/*)

* Detect and handle blobs in post embeds

* evert "Expand the available image resize fit modes"

This reverts commit 3390fbe4d091a04f9b67a2885eae19d025747fc4.

* Use fit: inside for post media

* Update tests

* Make the thumbnail optional in external embeds

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2022-12-14 14:28:59 -06:00
Daniel Holmgren
57bec34cf5
Post media (#400)
* adding in banners & some lex codegen improvements for server

* fix up spacing

* couple small fixes

* client codegen & remove media embed schemas

* hook up banners

* missed a couple of lexicons

* sketching out embed schemas

* change wrapper obj property name

* setting up indexing for post media

* adding to views

* codegen client

* some tests

* Update lexicons/app/bsky/feed/getAuthorFeed.json

Co-authored-by: devin ivy <devinivy@gmail.com>

* Update lexicons/app/bsky/feed/getPostThread.json

Co-authored-by: devin ivy <devinivy@gmail.com>

* Update lexicons/app/bsky/feed/getTimeline.json

Co-authored-by: devin ivy <devinivy@gmail.com>

* exhaustive check

Co-authored-by: devin ivy <devinivy@gmail.com>
2022-12-13 13:17:53 -06:00
Paul Frazee
3636bd19a8
A couple lexicon-related fixes (#405)
* Fix the generated type for encoding when the value is */* (close #391)

* Add 'toKnownError' test and fix named error for updateProfile

* Improve blob errors
2022-12-12 13:48:59 -06:00
Daniel Holmgren
a125f4e36d
Banners (#397)
* adding in banners & some lex codegen improvements for server

* fix up spacing

* couple small fixes

* client codegen & remove media embed schemas

* hook up banners

* missed a couple of lexicons
2022-12-12 10:18:50 -06:00
Daniel Holmgren
c74262299b
Rename upload blob NSID (#393)
* rename blob route

* Tweak to codegen after rebase

Co-authored-by: Paul Frazee <pfrazee@gmail.com>
2022-12-08 13:30:10 -06:00
Daniel Holmgren
2a4a1ec4d1
Various blob fixes (#394)
* fix byte streaming from mem blobstore

* Enlarge avatar maxSize to 300kb

* Skip the cache on empty files

Co-authored-by: Paul Frazee <pfrazee@gmail.com>
2022-12-08 13:26:05 -06:00
Paul Frazee
79c3f31f9e
Add 'avatar' to the outputs of views (#392)
* Add sharp to the dev-env deps

* Add 'avatar' to the outputs of views
2022-12-08 11:40:19 -06:00
Daniel Holmgren
d0b7497a08
File uploads & profile image support (#383)
* Setup interface for image processing

* Implement getInfo() on SharpImageProcessor

* Reorganize pds image processing code

* Implement initial resize() on SharpImageProcessor

* Test sharp image processor, apply a couple fixes

* Tidy

* wip

* improve repo processing & add blbos

* more blob processing

* work into routes & config

* testing out w profiles

* Implement initial pds image uri builder/signer/verifier

* pr fixup & tests

* streaming

* better streaming interface

* s3 interface

* fix s3 prefixes

* readable streams as xrpc inputs

* more tests + cleaning up apis

* tests for failed references

* Initial implementation of image processing server

* Update node types for node v18

* fix clone issue

* add getStream to blobstore

* fixing up tests & dev-env

* get img info on upload

* integrating img processing

* hex for img uri & fixing build errors

* improve streams

* hook up uris to getProfile

* pr feedback on tmp

* test on getprofile

* testing avatars

* fix snapshot after hmac changed

* quick cleanup

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2022-12-07 18:08:29 -06:00
Paul Frazee
d7b4697b2d
Add terms-of-service and privacy-policy links (#386)
* Add privacy-policy and terms-of-service to getAccountsConfig

* Add privacy policy and tos urls to dev environment
2022-12-06 13:05:05 -06:00
Paul Frazee
c966185f01
Lexicon tweaks: encoding and union $type discriminators (#382)
* Update LexXrpcBody encoding to always be a single string with comma-separated values

* Update lexicon union-type to use  as a discriminator, and add closed flag to establish if new union members may be added

* Fix to generated code for open unions
2022-12-05 09:38:51 -06:00
Paul Frazee
dd180f3dd1
Refactor lexicon definitions to reuse more shared types (#381)
* Factor out app.bsky.system.declRef

* Factor out app.bsky.actor.ref and app.bsky.feed.embed

* Factor out com.atproto.repo.strongRef and use app.bsky.actor.ref in more places

* Modify app.bsky.actor.ref to be light reference, add #withInfo for indexed info
2022-12-02 09:25:03 -06:00
Paul Frazee
f2aa6ddbea
Rework lexicon references to use a more consistent structure (#380) 2022-12-01 16:09:41 -06:00
Paul Frazee
63b9873bb1
Lexicon rework (#379)
* Implement types and validation for new NIH lexicon DSL

* Update xprc and xrpc-server to use new NIH lexicon DSL

* Simplification: dont allow lexicon definitions to be refs or ref arrays

* Add 'unknown' primitive type to NIH lexicon

* Implement client codegen for NIH Lexicon

* Implement server codegen for NIH Lexicon

* Add the 'datetime' primitive type to the NIH Lexicon

* Rework lexicons to use new NIH Lexicon DSL

* Readd tokens to code generation

* Update all code to use new NIH Lexicon (all tests passing!)

* Add imports from external files to lex-cli codegen

* Do not allow records, queries, or procedures outside of the main def

* Update packages/lex-cli/src/util.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Linter pass

* Add knownValue output to codegen

* Add comments to codegen

* Use enum and const in codegen

* Rework lexicon 'properties' structure

* Treat falsy query params as undefined

Co-authored-by: devin ivy <devinivy@gmail.com>

* Remove now-redundant param presence validation

Co-authored-by: devin ivy <devinivy@gmail.com>
2022-12-01 12:26:18 -06:00
Paul Frazee
47c69f5469
Update getPostThread to handle missing parents (#373) 2022-11-28 21:35:40 -06:00
Paul Frazee
ebf8e5c9ed
Add NotFoundError to getPostThread (#375) 2022-11-28 21:33:32 -06:00
devin ivy
134595e26e
Standardize limit schema across lexicons (#377) 2022-11-28 21:34:55 -05:00
devin ivy
fd359bada0
Use integers in lexicons (#367)
Use json schema integers rather than numbers where appropriate in lexicons
2022-11-23 16:13:37 -05:00
Daniel Holmgren
2d947be9c2
Allow updating scene profiles (#341)
allow updating scene profiles
2022-11-11 13:36:36 -06:00
Paul Frazee
a1bee406e2
Add app.bsky.graph.getAssertions() (#340)
* Add app.bsky.graph.getAssertions()

* Update getAssertions to support query by author or subject

* Revert accidental change to members test
2022-11-11 12:38:25 -06:00
devin ivy
8ab87ff1f7
Standard actor declarations in app.bsky (#338)
* Include scene trends in feeds

* Tidy

* Tidy

* Sweep app.bsky schemas for standard actor declarations, remove reference from com.atproto

* Replace declarationCid on follow, assertion, confirmation records

* Sweep pds for standard app.bsky actor declarations

* Fix dev-env

* Tidy after merge
2022-11-10 15:46:16 -05:00
devin ivy
c52b9f0771
Include scene trends in feeds (#335)
* Include scene trends in feeds

* Tidy

* Tidy
2022-11-10 14:26:57 -05:00
Daniel Holmgren
e6471a7419
Trend schemas (#333)
trend schemas
2022-11-09 14:44:46 -06:00
devin ivy
da8f055816
Align XRPC parameters with subset of json schema (#328)
* Model procedure for setting a vote on a post

* Implement app.bsky.feed.setVote() method

* Support json schema subset for parameters in lexicon and lex-cli

* Update lexicons w/ json schema parameters

* Use enum in getVotes parameters

* Support json schema subset for parameters in xrpc-server, ajv validation

* Include parameter enums in generated markdown

* Tidy

* Unify xrpc parameter md docs with inputs and outputs: show ts interfaces

* Tidy
2022-11-09 15:39:47 -05:00
devin ivy
1bd8ef1811
Implement setVote() method (#325)
* Model procedure for setting a vote on a post

* Implement app.bsky.feed.setVote() method

* Ensure setVote() no-ops when already in correct state
2022-11-09 15:24:35 -05:00
Paul Frazee
e1150465ab
Add 'member' to getProfile state output, and add some dev-env mock data (#332)
* Add a scene to the dev-env mock data

* Add 'member' to myState output of getProfile

* Await db writes in dev env mock data

Co-authored-by: devin ivy <devinivy@gmail.com>

Co-authored-by: devin ivy <devinivy@gmail.com>
2022-11-09 14:09:16 -06:00
Daniel Holmgren
58634c47d7
Scene membership (#322)
* schemas

* routes

* bugfix

* tests

* test fixups

* delcaration in member views

* sort notif output
2022-11-08 14:22:16 -06:00
devin ivy
b117ae4568
Move from likes to votes (#319)
* Model votes in lexicon, remove likes

* Model votes in db, setup notifs, fix schema

* Update pds logic, move likes to votes

* Codegen

* Update tests for votes

* Test vote filter by direction

* Fix dev-env
2022-11-08 11:02:41 -05:00
Paul Frazee
cd1b64e600
Add app.bsky.actor.getSuggestions (#318)
* Fix some lexicon filenames

* Add app.bsky.actor.getSuggestions
2022-11-08 09:40:52 -06:00
Alberto Roura
1a32d5664f
Added full stop to descriptions (#312)
Added full stop to description

Done for consistency across files and for potential future documentation.
2022-11-08 07:58:03 -06:00
Daniel Holmgren
b530a13d09
Scene creation & membership (#304)
* scene creation w unique handles

* wip

* user_dids -> did_handle

* invites -> assertion/confirmation

* do all the ops on scene create

* start tests + bug fixin

* user can make requests on behalf of scene

* accepting invites

* fix schema names

* get scene profiles

* return actor type in getProfile

* scene views test

* notifications test

* patch up a few db things

* rework handle & email noramlization

* codegen createscene errors
2022-11-07 14:33:33 -06:00
Paul Frazee
70c525f806
Rework post text slices to be objects (#311) 2022-11-04 18:26:33 -05:00
Paul Frazee
e61127111a
Split lexicons into more specific namespaces (#306)
* Segment lexicons into more specific namespaces and tighten up some naming conventions

* Update API to use new lexicons

* Update pds and dev-env to use new lexicons
2022-11-04 12:59:00 -05:00
Daniel Holmgren
7fdbb4dbb5
Username -> Handle (#303)
* schemas

* db

* atproto xrpc

* bsky xrpc

* got er done
2022-11-03 11:35:21 -05:00
Paul Frazee
9289ce15c6
Rename getHomeFeed to getTimeline to improve some semantic clarity (#300) 2022-11-02 16:52:25 -05:00