Commit Graph

83 Commits

Author SHA1 Message Date
Paul Frazee df6ed7d5c0 Add account preferences APIs (#1013)
* Add lexicons for account preferences

* Move prefs to app.bsky, codegen

* Setup model and services for user prefs

* Setup xrpc methods for prefs

* Test preferences, fixes

* Tidy

* Tidy

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-05-11 15:44:00 -05:00
Daniel Holmgren e32dc025e8 Mute lists (#986)
* possible schemas

* more lexicons

* codegen

* tables

* hook up processor

* getlist route

* add getListBlocks route

* migration

* didnt save something

* add reason + facets

* description facets

* get facets into db

* add to accnt deletion tests before i forget

* tests & bugfixes

* quickfix

* move defs to .graph

* add purpose

* remove unused actor defs

* add creator info to list view

* refactor to graph service

* lexicons

* impl

* writer methods

* bug in accnt service

* handle blobs

* tests

* fix typos

* tidy

* fix snap

* pr feedback

* missed drop index in migration

* remove listitem reason

* allow puts for lists

* make mute lists idempotent

* test mute list updates

* fix up

* fix err test

* update snap
2023-05-11 14:29:08 -05:00
Daniel Holmgren db1042cf0d Blocks (#922)
* block lexicons

* table schema

* block processing

* Actor-block migration

* Add blocked post lexicon to getPostThread

* getBlocks routes

* Add missing lexicons to bsky

* Add block state to actor views

* throw on blocked accounts in getAuthorFeed

* blcok state in profile views

* Update getPostThread to support block views, update existing test

* strip out blocks out oof timeline & popular

* more routes

* Test blocked posts in threads

* Support blocks on record embeds

* notifs

* Update pds test snapshots w/ block state

* fix search

* Update pds actor search snapshots w/ blocks

* test

* tests

* update lex

* fix up from returning blocking as record

* ci for branch

* fix build issue in bsky

* more blocks tests

* fix actor search test

* hook getBlocks up to router

* bugfix & actually test lol

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-04-28 19:36:02 -05:00
bnewbold aa20d20efb More label lexicons (#672)
* com.atproto.label.label lexicon

For persisting un-signed label objects in repositories.

There will likely be a handful of XRPC query endpoints in the
com.atproto.label namespace in the near future.

* lexicons: com.atproto.label.subscribeLabels

* lexicons: com.atproto.label.query

* lexicons: label updates post lex-refactor

* lexicons: rename label.query to label.queryLabels

* lexicons: queryLabels larger batch sizes; improve description

* lexicons: add labels to profiles, posts, notifications

* lexicons: add negation ('neg') flag to labels

* lexicon: label vals on moderation actions

This is a mechanism to create (or negate) labels as part of a moderation
action. The fields are optional, so this is backwards compatible, and if
we wanted to embed entire labels in the future (instead of just "val"
strings), that would be possible using a different field name.

* lexicons: update label.label to label.defs#label

* codegen

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2023-04-11 12:44:19 -05:00
Daniel Holmgren eb488b96f5 lex refactor hot fixes (#745)
* filter embeds from get popular

* Truncate profile info to satisfy validation (#746)

* Remove trailing replacement character when utf8-truncating (#747)

* Truncate profile info to satisfy validation

* Fix utf8 truncation w/ replacement character

* filter replies

* delete embeds from records on getPopular

* Update profile display name and description lengths to be based on graphemes (#748)

* @atproto api v0.2.1

* Tidy

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-04-04 15:56:53 -04:00
Daniel Holmgren 7f008c05a0 Lexicon refactor (#658)
* remove return in test

* couple of fixups in other pacakges

* Add dummy checks to declaration and follow app migrations, remove paranoid join

* update db nsid migration

* Ensure there are writes in follow app migration

* Add dumy check to votes-to-likes app migration, tidy

* Ensure there are writes in vote-to-like app migration

* update migration name

* pr feedback

* count utf8 & grapheme length

* add maxUtf8

* siwtch max semantics

* plural

* update post schema

* added bytes & cid refs

* add ipld<>json

* fixin up a could tings

* Add app.bsky.richtext.facet, replace post entities with facets

* plural actors

* wip

* Setup backlinks table on pds

* wip

* send & recieve cids/bytes with xrpc

* Track backlinks when indexing records on pds

* handle ipld vals in xrpc server

* added cids & bytes to codegen

* In createRecord, add deletions to avoid duplicate likes/follows/reposts

* Tests and fixes for prevention of dupe follows, likes, reposts

* Backlink migration tidy

* cleanup dag json parser

* Fix dupe backlink inserts

* Tidy

* blob refs + codegen

* Make profile displayName optional

* Test view and updateProfile for empty display name

* working into pds

* Make aggregate counts optional on post and profile views

* Make viewer state optional on post view for consistency

* Remove deprecated myState field on profile view

* Tidy repo method descriptions

* tests & types & fixes

* Implementation and tests for putRecord

* Remove updateProfile method

* Update repo service so that head can be taken for update externally

* Lex updates for compare-and-swap records/commits

* Add error to lex for bad repo compare-and-swaps

* Improve update-at-head thru repo service

* common package

* Implement and test compare-and-swaps on repo write methods

* Use lex discriminator for applyWrites

* Remove post entity/facet index

* Update lex descriptions to clarify repo write semantics

* Make deleteRecord idempotent w/ tests

* cleanup

* fix things up

* adding more formats

* tests

* updating schema

* Only generate tid rkeys on pds, support literal rkeys on client

* Add backlink indexes

* Update format of post embed views, fix external uri validation

* fixing up tests

* Include embeds on record embeds

* cleanup

* Notify users when they are quoted

* Remove determineRkey indirection

* fix api tests

* support concatenated cbor

* integrating to server

* re-enable tests

* fix up tests

* Thread compare-and-swaps down into repo service rather than use pinned storage

* Tidy

* Update packages/common/tests/ipld-multi.test.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Update packages/lexicon/src/validators/formats.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* pr feedback

* pr feedback

* Add postgres-specific migration path for missing profile display names

* Tidy/clarify deep embeds

* Tidy

* rm unused escape

* decrease crud race count

* update subscribeRepos lexicon

* Fix applyWrite lexicon re: collection fields

* sign post event type

* update cids & bytes json encoding

* update lex blob & cid-link types

* updated codegen & pds

* number -> float

* missed a couple

* remove old image constraints

* pr feedback + descripts

* no hardcoded port numbers

* remove separate tooLarge evt

* fix dumb build error

* fixin gup lex + xrpc server

* better parsing of message types

* dont mutate body in subscription

* bugfix in subscription

* rm commented out code

* init feature branch

* undo

* Remove old lexicons

* Remove creator from profile view

* wip

* rework seqs

* fixed up tests

* bug fixing

* sequence handles & notify in dbTxn

* tidy

* update lex to include times

* test syncing handle changes

* one more fix

* handle too big evts

* dont thread sequencer through everything

* Split common into server vs web-friendly versions

* Make lexicon, identifier web-safe using common-web

* Switch api package to be a browser build, fix identifier package for browser bundling

* Fix pds and repo for lexicon package changes, tidy

* Make common-web a browser build, tidy

* fixing up deps

* fix up test

* turn off caching in actions

* Standardize repo write interfaces around repo input

* Update repo write endpoints for repo input field

* Remove scene follows during app migration

* API package updates (#712)

* Add bsky agent and various sugars to the api package

* Add richtext library to api package

* Update richtext to use facets and deprecate entities

* Update richtext to use utf8 indices

* Richtext converts deprecated entity indices from utf16 locations to utf8 locations

* Add note about encodings in the lexicon

* Add RichText facet detection

* Remove dead code

* Add deprecation notices to lexicons

* Usability improvements to RichText

* Update the api package readme

* Add RichText#detectFacetsWithoutResolution

* Add upsertProfile to bsky-agent

* Update packages/pds/src/api/com/atproto/repo/applyWrites.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* pr feedback

* fix flaky timing streaming tests

* simplify emptyPromise

* fixed up open handles

* fix missed repo syntax

* fix error in test from fkey constraint

* fix another api agent bug

* Embed consistency, add complex record embed

* Tidy embed lex descriptions

* rename pg schemas

* use swc for jest

* fix up deps

* cleanup

* Update pds indexing, views, tests for complex record embeds

* fixing up profile view semantics

* wip

* update snaps

* Rename embed.complexRecord to embed.recordWithMedia

* Tidy aroud record w/ media embeds

* Add grapheme utilities to api RichText (#720)

Co-authored-by: dholms <dtholmgren@gmail.com>

* Fix: app.bsky.feed.getPostThread#... to app.bsky.feed.defs#... (#726)

* Update bskyagent to use repo param

* Minor typing fix

* Add exports to api package: blobref & lex/json converters (#727)

* Add exports to api package: BlobRef & lex/json converters

* Add an example react-native fetch handler

* Switch all lingering references of recordRef to strongRef

* Update lexicon for richtext facets to have multiple features, byte slice rather than text slice

* Implement multi-feature richtext facets on pds

* Update api package to use updated richtext facets

* Minor fixes to admin repo/record views

* Fix app migration exports, remove old app migration

* Fix: sort richtext facets so they can render correctly

* Disable app migration dummy checks that don't work on live deploy

* Optimize lex de/serialization using simple checks

* Tidy comment typos

* App migration to cleanup notifications for likes, follows, old scene notifs

* Fix notification reason for change from vote to like

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
2023-03-31 13:34:51 -04:00
devin ivy 9c119fb9c6 Allow unsetting profile images and description in updateProfile (#554)
* Add support for nullable on lex objects

* Support clearing fields in updateProfile lexicon

* Support clearing fields in updateProfile method
2023-02-16 14:32:25 -05:00
devin ivy 850b54fad6 Add following state to profile/actor views, reorg views (#539)
* Add lexicon for getProfiles

* Split out presentation of profiles, implement getProfiles, test

* Cull actor/profile view lexicons down to three variants

* Map viewer in profile view, mark todos, lex tweak

* Apply basic profile view to methods, lex fix

* Switch searchTypeahead to actor-with-info view

* Switch getVotes to actor-with-info view

* Switch getMutes to actor-with-info view

* Switch getFollows and getFollowers to actor-with-info view

* Switch getRepostedBy to actor-with-info view

* Switch feed and notification views to use updated actor-with-info view
2023-02-16 11:40:05 -05:00
devin ivy 71d812abe6 Get profiles xrpc method (#532)
* Add lexicon for getProfiles

* Split out presentation of profiles, implement getProfiles, test

* Move ArrayEl type helper into common
2023-02-09 11:11:46 -05:00
Daniel Holmgren 396698f218 Filter out self & follows from suggestions (#498)
dont suggest self or follows
2023-02-02 16:48:41 -05:00
Daniel Holmgren 05855fcff3 Loosening image constraints (#503)
* loosening img constraints

* fixing up tests
2023-01-30 18:59:19 -05:00
Daniel Holmgren 6f780bf670 Remove scenes (#483)
* wip

* wip

* remove getAssertions & fix up notifs test

* remove did param on updateProfile
2023-01-26 15:38:45 -06:00
devin ivy e9f5f23edc Fix lexicon validation, do not allow undefined when required (#457)
* Fix lexicon validation, do not allow undefined when required

* Make term optional for actor search/typeahead
2023-01-05 12:04:31 -05:00
devin ivy aa626c3dc4 User muting (#445)
* Procedures and view for user muting

* Update profile and feed/thread methods for muting, update tests

* Tidy

* Test muting for timeline, actor feed, thread, profile views, misc fixes

* Test mute views, misc fixes

* Add missing file

* Add per-user notification index

* Omit mention and reply notifications from muted users

* Test timing leeway
2022-12-30 14:27:48 -05:00
Paul Frazee 1ede96c4ad Improve image quality (#442)
* Increase the allowed dimenions of all images

* Increase the size and quality of served images

* Update tests for larger res
2022-12-26 15:51:54 -06:00
Paul Frazee 3636bd19a8 A couple lexicon-related fixes (#405)
* Fix the generated type for encoding when the value is */* (close #391)

* Add 'toKnownError' test and fix named error for updateProfile

* Improve blob errors
2022-12-12 13:48:59 -06:00
Daniel Holmgren a125f4e36d Banners (#397)
* adding in banners & some lex codegen improvements for server

* fix up spacing

* couple small fixes

* client codegen & remove media embed schemas

* hook up banners

* missed a couple of lexicons
2022-12-12 10:18:50 -06:00
Daniel Holmgren 2a4a1ec4d1 Various blob fixes (#394)
* fix byte streaming from mem blobstore

* Enlarge avatar maxSize to 300kb

* Skip the cache on empty files

Co-authored-by: Paul Frazee <pfrazee@gmail.com>
2022-12-08 13:26:05 -06:00
Paul Frazee 79c3f31f9e Add 'avatar' to the outputs of views (#392)
* Add sharp to the dev-env deps

* Add 'avatar' to the outputs of views
2022-12-08 11:40:19 -06:00
Daniel Holmgren d0b7497a08 File uploads & profile image support (#383)
* Setup interface for image processing

* Implement getInfo() on SharpImageProcessor

* Reorganize pds image processing code

* Implement initial resize() on SharpImageProcessor

* Test sharp image processor, apply a couple fixes

* Tidy

* wip

* improve repo processing & add blbos

* more blob processing

* work into routes & config

* testing out w profiles

* Implement initial pds image uri builder/signer/verifier

* pr fixup & tests

* streaming

* better streaming interface

* s3 interface

* fix s3 prefixes

* readable streams as xrpc inputs

* more tests + cleaning up apis

* tests for failed references

* Initial implementation of image processing server

* Update node types for node v18

* fix clone issue

* add getStream to blobstore

* fixing up tests & dev-env

* get img info on upload

* integrating img processing

* hex for img uri & fixing build errors

* improve streams

* hook up uris to getProfile

* pr feedback on tmp

* test on getprofile

* testing avatars

* fix snapshot after hmac changed

* quick cleanup

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2022-12-07 18:08:29 -06:00
Paul Frazee dd180f3dd1 Refactor lexicon definitions to reuse more shared types (#381)
* Factor out app.bsky.system.declRef

* Factor out app.bsky.actor.ref and app.bsky.feed.embed

* Factor out com.atproto.repo.strongRef and use app.bsky.actor.ref in more places

* Modify app.bsky.actor.ref to be light reference, add #withInfo for indexed info
2022-12-02 09:25:03 -06:00
Paul Frazee f2aa6ddbea Rework lexicon references to use a more consistent structure (#380) 2022-12-01 16:09:41 -06:00
Paul Frazee 63b9873bb1 Lexicon rework (#379)
* Implement types and validation for new NIH lexicon DSL

* Update xprc and xrpc-server to use new NIH lexicon DSL

* Simplification: dont allow lexicon definitions to be refs or ref arrays

* Add 'unknown' primitive type to NIH lexicon

* Implement client codegen for NIH Lexicon

* Implement server codegen for NIH Lexicon

* Add the 'datetime' primitive type to the NIH Lexicon

* Rework lexicons to use new NIH Lexicon DSL

* Readd tokens to code generation

* Update all code to use new NIH Lexicon (all tests passing!)

* Add imports from external files to lex-cli codegen

* Do not allow records, queries, or procedures outside of the main def

* Update packages/lex-cli/src/util.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Linter pass

* Add knownValue output to codegen

* Add comments to codegen

* Use enum and const in codegen

* Rework lexicon 'properties' structure

* Treat falsy query params as undefined

Co-authored-by: devin ivy <devinivy@gmail.com>

* Remove now-redundant param presence validation

Co-authored-by: devin ivy <devinivy@gmail.com>
2022-12-01 12:26:18 -06:00
devin ivy 134595e26e Standardize limit schema across lexicons (#377) 2022-11-28 21:34:55 -05:00
devin ivy fd359bada0 Use integers in lexicons (#367)
Use json schema integers rather than numbers where appropriate in lexicons
2022-11-23 16:13:37 -05:00
Daniel Holmgren 2d947be9c2 Allow updating scene profiles (#341)
allow updating scene profiles
2022-11-11 13:36:36 -06:00
devin ivy 8ab87ff1f7 Standard actor declarations in app.bsky (#338)
* Include scene trends in feeds

* Tidy

* Tidy

* Sweep app.bsky schemas for standard actor declarations, remove reference from com.atproto

* Replace declarationCid on follow, assertion, confirmation records

* Sweep pds for standard app.bsky actor declarations

* Fix dev-env

* Tidy after merge
2022-11-10 15:46:16 -05:00
devin ivy da8f055816 Align XRPC parameters with subset of json schema (#328)
* Model procedure for setting a vote on a post

* Implement app.bsky.feed.setVote() method

* Support json schema subset for parameters in lexicon and lex-cli

* Update lexicons w/ json schema parameters

* Use enum in getVotes parameters

* Support json schema subset for parameters in xrpc-server, ajv validation

* Include parameter enums in generated markdown

* Tidy

* Unify xrpc parameter md docs with inputs and outputs: show ts interfaces

* Tidy
2022-11-09 15:39:47 -05:00
Paul Frazee e1150465ab Add 'member' to getProfile state output, and add some dev-env mock data (#332)
* Add a scene to the dev-env mock data

* Add 'member' to myState output of getProfile

* Await db writes in dev env mock data

Co-authored-by: devin ivy <devinivy@gmail.com>

Co-authored-by: devin ivy <devinivy@gmail.com>
2022-11-09 14:09:16 -06:00
Paul Frazee cd1b64e600 Add app.bsky.actor.getSuggestions (#318)
* Fix some lexicon filenames

* Add app.bsky.actor.getSuggestions
2022-11-08 09:40:52 -06:00
Alberto Roura 1a32d5664f Added full stop to descriptions (#312)
Added full stop to description

Done for consistency across files and for potential future documentation.
2022-11-08 07:58:03 -06:00
Daniel Holmgren b530a13d09 Scene creation & membership (#304)
* scene creation w unique handles

* wip

* user_dids -> did_handle

* invites -> assertion/confirmation

* do all the ops on scene create

* start tests + bug fixin

* user can make requests on behalf of scene

* accepting invites

* fix schema names

* get scene profiles

* return actor type in getProfile

* scene views test

* notifications test

* patch up a few db things

* rework handle & email noramlization

* codegen createscene errors
2022-11-07 14:33:33 -06:00
Paul Frazee e61127111a Split lexicons into more specific namespaces (#306)
* Segment lexicons into more specific namespaces and tighten up some naming conventions

* Update API to use new lexicons

* Update pds and dev-env to use new lexicons
2022-11-04 12:59:00 -05:00