16 Commits

Author SHA1 Message Date
Samuel Newman
2676206e42
Pinned posts (#2771)
* pinned posts lexicon

* codegen

* change lexicon, different approach

* codegen 2

* dataplane db migration

* move pinned post lexicon to right place

* add pinned posts optionally to getAuthorFeed

* remove type modification

* Clarify naming, add viewer state, add tests

* return pinnedPost with profileViewDetailed

* allow pinned replies in `posts_and_author_threads`

* clearer variable naming

* annotate type of `items`

* boolean --> varchar

* reuse authorDid in viewerPinned

* simplify test

* make pinned post not top post in test

* update snapshot

* changeset

---------

Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: dholms <dtholmgren@gmail.com>
2024-09-26 18:26:45 -05:00
devin ivy
af7d3912a3
Starter packs (#2553)
* lexicon: initial starter pack record lexicons

* lexicon: first pass on views for starter packs

* lexicon: add starter pack query endpoints, misc fixes

* lexicon: add starter pack related fields to profile views

* lexicon: codegen

* api prerelease

* bsky: protos for starter packs

* appview: implement starter pack endpoints in dataplane, protos update

* appview: starter pack hydration and view, mock dataplane indexing

* appview: wire-up starter pack graph endpoints

* appview: test basic starter packs functionality, update dev-env

* fix snaps for profile createdAt

* appview: fix list item count for starter packs in dataplane

* fix ozone snaps, tidy

* appview: reorg list aggs for starter packs

* appview: starter pack lexicon and proto tweaks, move around aggregations

* appview: fix views and snaps

* api prerelease

* fix snaps

* appview: misc proto updates for starter packs, snaps

* appview: add starter pack counts to profile detail

* build

* api prerelease

* pds: starter pack slur detection

* lexicon: starter pack notifications and batch fetch

* appview: starter pack notification and batch fetch

* appview: add starterpack-joined notif to dataplane and test

* api prerelease

* ozone: fix snaps

* pds build

*  Proxy starterpack requests through ozone and include takedowns (#2594)

*  Proxy starterpack requests through ozone and include takedowns

* 🧹 Cleanup

*  Commit snapshots

*  Update snapshot

* lexicon: move starterpacks back to TIDs

* changeset

---------

Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
2024-06-24 21:04:33 -04:00
bnewbold
9579bec720
update descriptions in Lexicons (#2110)
* lexicons: update descriptions in com.atproto.*

* lexicons: update descriptions in app.bsky.*

* Apply suggestions from code review

Thanks Emily!

Co-authored-by: Emily Liu <emilyliu7321@gmail.com>

* codegen description-only Lexicon updates

---------

Co-authored-by: Emily Liu <emilyliu7321@gmail.com>
2024-02-11 16:06:58 -08:00
Emily Liu
59f70dbe80
Tweak lexicon descriptions (#1859)
* Tweak lexicon descriptions

* Tweak birthdate

* Update lexicons/app/bsky/actor/getProfiles.json

Co-authored-by: devin ivy <devinivy@gmail.com>

* Regenerate packages from lexicons

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-11-20 17:41:25 -05:00
Paul Frazee
ab50816461
Self-labeling (#1444)
* Add self-label schemas

* switch around array and union for self-labels

* apply self-labels to post and profile views

* test for self labels in pds

* apply self-labels to post and profile views in bsky appview

* test for self labels in bsky appview

* update pds proxied test snapshots

* Add support for self-labels to the mdoeration sdk

* Disable unknown-labeler filtering until 3P support is added

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-08-09 19:06:29 -04:00
Daniel Holmgren
eb488b96f5
lex refactor hot fixes (#745)
* filter embeds from get popular

* Truncate profile info to satisfy validation (#746)

* Remove trailing replacement character when utf8-truncating (#747)

* Truncate profile info to satisfy validation

* Fix utf8 truncation w/ replacement character

* filter replies

* delete embeds from records on getPopular

* Update profile display name and description lengths to be based on graphemes (#748)

* @atproto api v0.2.1

* Tidy

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-04-04 15:56:53 -04:00
Daniel Holmgren
7f008c05a0
Lexicon refactor (#658)
* remove return in test

* couple of fixups in other pacakges

* Add dummy checks to declaration and follow app migrations, remove paranoid join

* update db nsid migration

* Ensure there are writes in follow app migration

* Add dumy check to votes-to-likes app migration, tidy

* Ensure there are writes in vote-to-like app migration

* update migration name

* pr feedback

* count utf8 & grapheme length

* add maxUtf8

* siwtch max semantics

* plural

* update post schema

* added bytes & cid refs

* add ipld<>json

* fixin up a could tings

* Add app.bsky.richtext.facet, replace post entities with facets

* plural actors

* wip

* Setup backlinks table on pds

* wip

* send & recieve cids/bytes with xrpc

* Track backlinks when indexing records on pds

* handle ipld vals in xrpc server

* added cids & bytes to codegen

* In createRecord, add deletions to avoid duplicate likes/follows/reposts

* Tests and fixes for prevention of dupe follows, likes, reposts

* Backlink migration tidy

* cleanup dag json parser

* Fix dupe backlink inserts

* Tidy

* blob refs + codegen

* Make profile displayName optional

* Test view and updateProfile for empty display name

* working into pds

* Make aggregate counts optional on post and profile views

* Make viewer state optional on post view for consistency

* Remove deprecated myState field on profile view

* Tidy repo method descriptions

* tests & types & fixes

* Implementation and tests for putRecord

* Remove updateProfile method

* Update repo service so that head can be taken for update externally

* Lex updates for compare-and-swap records/commits

* Add error to lex for bad repo compare-and-swaps

* Improve update-at-head thru repo service

* common package

* Implement and test compare-and-swaps on repo write methods

* Use lex discriminator for applyWrites

* Remove post entity/facet index

* Update lex descriptions to clarify repo write semantics

* Make deleteRecord idempotent w/ tests

* cleanup

* fix things up

* adding more formats

* tests

* updating schema

* Only generate tid rkeys on pds, support literal rkeys on client

* Add backlink indexes

* Update format of post embed views, fix external uri validation

* fixing up tests

* Include embeds on record embeds

* cleanup

* Notify users when they are quoted

* Remove determineRkey indirection

* fix api tests

* support concatenated cbor

* integrating to server

* re-enable tests

* fix up tests

* Thread compare-and-swaps down into repo service rather than use pinned storage

* Tidy

* Update packages/common/tests/ipld-multi.test.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Update packages/lexicon/src/validators/formats.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* pr feedback

* pr feedback

* Add postgres-specific migration path for missing profile display names

* Tidy/clarify deep embeds

* Tidy

* rm unused escape

* decrease crud race count

* update subscribeRepos lexicon

* Fix applyWrite lexicon re: collection fields

* sign post event type

* update cids & bytes json encoding

* update lex blob & cid-link types

* updated codegen & pds

* number -> float

* missed a couple

* remove old image constraints

* pr feedback + descripts

* no hardcoded port numbers

* remove separate tooLarge evt

* fix dumb build error

* fixin gup lex + xrpc server

* better parsing of message types

* dont mutate body in subscription

* bugfix in subscription

* rm commented out code

* init feature branch

* undo

* Remove old lexicons

* Remove creator from profile view

* wip

* rework seqs

* fixed up tests

* bug fixing

* sequence handles & notify in dbTxn

* tidy

* update lex to include times

* test syncing handle changes

* one more fix

* handle too big evts

* dont thread sequencer through everything

* Split common into server vs web-friendly versions

* Make lexicon, identifier web-safe using common-web

* Switch api package to be a browser build, fix identifier package for browser bundling

* Fix pds and repo for lexicon package changes, tidy

* Make common-web a browser build, tidy

* fixing up deps

* fix up test

* turn off caching in actions

* Standardize repo write interfaces around repo input

* Update repo write endpoints for repo input field

* Remove scene follows during app migration

* API package updates (#712)

* Add bsky agent and various sugars to the api package

* Add richtext library to api package

* Update richtext to use facets and deprecate entities

* Update richtext to use utf8 indices

* Richtext converts deprecated entity indices from utf16 locations to utf8 locations

* Add note about encodings in the lexicon

* Add RichText facet detection

* Remove dead code

* Add deprecation notices to lexicons

* Usability improvements to RichText

* Update the api package readme

* Add RichText#detectFacetsWithoutResolution

* Add upsertProfile to bsky-agent

* Update packages/pds/src/api/com/atproto/repo/applyWrites.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* pr feedback

* fix flaky timing streaming tests

* simplify emptyPromise

* fixed up open handles

* fix missed repo syntax

* fix error in test from fkey constraint

* fix another api agent bug

* Embed consistency, add complex record embed

* Tidy embed lex descriptions

* rename pg schemas

* use swc for jest

* fix up deps

* cleanup

* Update pds indexing, views, tests for complex record embeds

* fixing up profile view semantics

* wip

* update snaps

* Rename embed.complexRecord to embed.recordWithMedia

* Tidy aroud record w/ media embeds

* Add grapheme utilities to api RichText (#720)

Co-authored-by: dholms <dtholmgren@gmail.com>

* Fix: app.bsky.feed.getPostThread#... to app.bsky.feed.defs#... (#726)

* Update bskyagent to use repo param

* Minor typing fix

* Add exports to api package: blobref & lex/json converters (#727)

* Add exports to api package: BlobRef & lex/json converters

* Add an example react-native fetch handler

* Switch all lingering references of recordRef to strongRef

* Update lexicon for richtext facets to have multiple features, byte slice rather than text slice

* Implement multi-feature richtext facets on pds

* Update api package to use updated richtext facets

* Minor fixes to admin repo/record views

* Fix app migration exports, remove old app migration

* Fix: sort richtext facets so they can render correctly

* Disable app migration dummy checks that don't work on live deploy

* Optimize lex de/serialization using simple checks

* Tidy comment typos

* App migration to cleanup notifications for likes, follows, old scene notifs

* Fix notification reason for change from vote to like

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
2023-03-31 13:34:51 -04:00
devin ivy
850b54fad6
Add following state to profile/actor views, reorg views (#539)
* Add lexicon for getProfiles

* Split out presentation of profiles, implement getProfiles, test

* Cull actor/profile view lexicons down to three variants

* Map viewer in profile view, mark todos, lex tweak

* Apply basic profile view to methods, lex fix

* Switch searchTypeahead to actor-with-info view

* Switch getVotes to actor-with-info view

* Switch getMutes to actor-with-info view

* Switch getFollows and getFollowers to actor-with-info view

* Switch getRepostedBy to actor-with-info view

* Switch feed and notification views to use updated actor-with-info view
2023-02-16 11:40:05 -05:00
devin ivy
71d812abe6
Get profiles xrpc method (#532)
* Add lexicon for getProfiles

* Split out presentation of profiles, implement getProfiles, test

* Move ArrayEl type helper into common
2023-02-09 11:11:46 -05:00
Daniel Holmgren
05855fcff3
Loosening image constraints (#503)
* loosening img constraints

* fixing up tests
2023-01-30 18:59:19 -05:00
Paul Frazee
1ede96c4ad
Improve image quality (#442)
* Increase the allowed dimenions of all images

* Increase the size and quality of served images

* Update tests for larger res
2022-12-26 15:51:54 -06:00
Daniel Holmgren
a125f4e36d
Banners (#397)
* adding in banners & some lex codegen improvements for server

* fix up spacing

* couple small fixes

* client codegen & remove media embed schemas

* hook up banners

* missed a couple of lexicons
2022-12-12 10:18:50 -06:00
Daniel Holmgren
2a4a1ec4d1
Various blob fixes (#394)
* fix byte streaming from mem blobstore

* Enlarge avatar maxSize to 300kb

* Skip the cache on empty files

Co-authored-by: Paul Frazee <pfrazee@gmail.com>
2022-12-08 13:26:05 -06:00
Daniel Holmgren
d0b7497a08
File uploads & profile image support (#383)
* Setup interface for image processing

* Implement getInfo() on SharpImageProcessor

* Reorganize pds image processing code

* Implement initial resize() on SharpImageProcessor

* Test sharp image processor, apply a couple fixes

* Tidy

* wip

* improve repo processing & add blbos

* more blob processing

* work into routes & config

* testing out w profiles

* Implement initial pds image uri builder/signer/verifier

* pr fixup & tests

* streaming

* better streaming interface

* s3 interface

* fix s3 prefixes

* readable streams as xrpc inputs

* more tests + cleaning up apis

* tests for failed references

* Initial implementation of image processing server

* Update node types for node v18

* fix clone issue

* add getStream to blobstore

* fixing up tests & dev-env

* get img info on upload

* integrating img processing

* hex for img uri & fixing build errors

* improve streams

* hook up uris to getProfile

* pr feedback on tmp

* test on getprofile

* testing avatars

* fix snapshot after hmac changed

* quick cleanup

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2022-12-07 18:08:29 -06:00
Paul Frazee
63b9873bb1
Lexicon rework (#379)
* Implement types and validation for new NIH lexicon DSL

* Update xprc and xrpc-server to use new NIH lexicon DSL

* Simplification: dont allow lexicon definitions to be refs or ref arrays

* Add 'unknown' primitive type to NIH lexicon

* Implement client codegen for NIH Lexicon

* Implement server codegen for NIH Lexicon

* Add the 'datetime' primitive type to the NIH Lexicon

* Rework lexicons to use new NIH Lexicon DSL

* Readd tokens to code generation

* Update all code to use new NIH Lexicon (all tests passing!)

* Add imports from external files to lex-cli codegen

* Do not allow records, queries, or procedures outside of the main def

* Update packages/lex-cli/src/util.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* Linter pass

* Add knownValue output to codegen

* Add comments to codegen

* Use enum and const in codegen

* Rework lexicon 'properties' structure

* Treat falsy query params as undefined

Co-authored-by: devin ivy <devinivy@gmail.com>

* Remove now-redundant param presence validation

Co-authored-by: devin ivy <devinivy@gmail.com>
2022-12-01 12:26:18 -06:00
Paul Frazee
e61127111a
Split lexicons into more specific namespaces (#306)
* Segment lexicons into more specific namespaces and tighten up some naming conventions

* Update API to use new lexicons

* Update pds and dev-env to use new lexicons
2022-11-04 12:59:00 -05:00