* lexicon: initial lexicons for video embeds in bsky app
* lexicon: fix video caption file size limit
* codegen
* appview: stub out video embed view logic
* api prerelease
* api prerelease
* lexicon: video upload/processing lexicons
* tidy
* lexicon: app.bsky.video lexicons for uploads
* codegen
* api prerelease
* appview: present video embeds on posts
* appview: snaps
* changeset
* appview: fix wiring of video url config
* Add new postgate lex, hiddeReplies to threadgate, codegen
* Add protobufs
* Add to mock dataplane
* Add matching postgate method to feed hydration methods
* Add to getRecord
* Add to HydrationState
* Fix typo
* Add to mergeStates, fetch embeds in threads
* Integrate into embed views
* Add test for QPs in threads
* Add feed test
* Fix naming convention in protos
* Add #viewRemoved record view, rename postgate.json
* Integrate new view
* Filter hidden replies from feeds
* Filter out replies at the handler level, do not filter for author feeds
* Fix lint
* Move hidden reply check to view layer
* Reduce, reuse, recycle
* Rename to lowercase
* Rename layer vars
* Add quote gate props to postgate (#2693)
* Add quote gate props to postgate
* Consistent naming
* Fix record structure
* Codegen
* Show hidden replies in author feed
* Allow reposts of hidden replies
* Lex and codegen
* Add violates_quote_gate to proto
* Consistent naming, codegen
* Integrate violatesQuotegate and canQuotepost
* Remove rules, codegen
* Hydrate all postgates for all requested posts
* Match other impl
* Add test, need to split these out
* Format
* Hydrate first nested embeds too
* Add postgate test suite
* Add violatesQuoteGate to dataplane
* Ingest and set violatesQuoteGate, return on meta
* Return removed embed for quotes that violate gate
* Add test
* Dedupe URIs before fetching postgates
* Update snaps
* Snap
* Format
* Updating naming conventions for postgate-related attributes
* Correct naming
* Consistency
* Proto too
* Rename to viewDetached
* Codegen
* Rename everything
* Codegen
* Quotes that violate a quote gate can still be quoted themselves
* Couple more renames
* Snaps
* Ensure reply ref is tombstoned for hidden replies
* Split out hidden replies tests and create fresh fixture
* Hydrate threadgates for reply notifications, filter hidden replies
* Remove snap
* Add flaky test
* Rename violatesEmbeddingRules
* Fix flaky test
* Only write to db if violatesEmbeddingRules is true
* DRY up post uri -> gate uri logic
* isThreadgateListRule
* Don't share users object between tests
* No pascal
* Remove default params
* Find -> some
* canQuotepost -> canEmbed, remove optional
* Fix quoteee typo
* await follows
* Throw in post uri -> gate utils
* Ensure fetch threadgates for reply roots
* Don't hydrate threadgates twice
* DRY up uri -> did parsing
* Clean up parsePostgate logic
* Format
* Revert change
* Revert change
* Replace a couple more uri->did conversions
* Only filter replies from feeds if viewer hid them
* Revert, filter out replies that are hidden from feeds
* Remove old test
* Replace uri->did util
* Revert change to unused file
* Only validatePostEmbed and check postgates for post records
* Ensure notifications aren't generated down a hidden reply chain
* Changeset
* Cleanup
* Fix notification filtering logic
* Simplify
* Don't notify for invalid embeds
* Use new APIs
* Add hasPostGate and hasThreadGate flags from dataplane
* Only fetch postgates if post has one
* Only fetch threadgates if post has one or was deleted
* Remove notification filtering
* Don't hydrate threadgates for notifications
* Move hidden replies in feeds to match block handling
* Do no filtering of hidden replies in feeds
* Revert "Don't hydrate threadgates for notifications"
This reverts commit 1dcec0b239a7b9d6800427b26b8ba3e6a54210f9.
* Revert "Remove notification filtering"
This reverts commit 1e7069dfd809d1f18e9f05fd1d422e7399aa1bb0.
* Filter notifications for OP only
* Add additional check to hidden replies test
* Move noty filter logic into method handler
* Update .changeset/perfect-parrots-appear.md
Co-authored-by: devin ivy <devinivy@gmail.com>
* Update packages/bsky/tests/seed/postgates.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* Another structuredClone
* Update packages/bsky/src/hydration/hydrator.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* Better comment
* Update packages/bsky/src/data-plane/server/indexing/plugins/post.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* Regen protos to match dataplane
* Update quotes snap to include embeddingDisabled
* Clarify usage of post uri -> gate utils
---------
Co-authored-by: devin ivy <devinivy@gmail.com>
* lexicon: add post counts to embed#viewRecord
* appview: hydrate and present aggregations on post embeds
* update snaps for aggregations on post embeds
* adding some basic views
* feed hydration, add blocks to post hydration
* pass over notification hydration
* tidy
* merge
* implement getProfile
* hydrate post aggregation and viewer state
* fix
* fix codegen
* get some tests passing!
* add takedowns & some like bugfixing
* all profile tests passing!
* likes test
* follow endpoints using data plane
* reorg follow block rules
* reposts
* post views!
* implement getList w/ dataplane caveat
* adjust dataplane getListMembers to return listitem uris
* implement getListMutes and -Blocks w/ dataplane
* suggestions
* timeline
* misc view fixes
* view fixes for mutes, self-mute/block
* author feed
* feed gen routes
* tidy
* misc block/mute fixes
* list feed & actor likes
* implement getLists, fix some empty cursors
* implement getMutes, empty profile description fix
* implement getBlocks, block application fix
* implement getSuggestedFollowsByActor, needs some fixes
* feed generation
* search routes
* threads
* tidy
* fix some snaps
* fix getSuggestedFollowsByActor
* implement listNotifications
* implement getUnreadCount w/ dataplane
* implement notifications.updateSeen w/ dataplane
* 3rd party blocking tests
* blocked profile viewer
* add search mocks
* refactor getFeed
* createPipeline -> createPipelineNew
* basic replygating functionality on dataplane w/o filtering violating replies
* hack threadgates into dataplane, apply gates
* deterministic thread orders in dataplane
* misc cleanup around dataplane
* upgrade typescript to v5.3
* update typescript linter deps
* sync bsky proto, codegen
* update dataplane, sync with bsky proto updates
* remove indexer, ingester, daemon, moderation services from appview
* convert more bsky internals to dataplane, remove custom feedgens, implement mute/unmuting in mock dataplane
* remove bsky services. db and indexing logic into mock dataplane.
* remove tests not needed by appview v2, misc reorg
* add basic in-mem repo subscription to dataplane mock
* fix dev-env, bsky tests, bsky build
* cull bsky service entrypoint
* add bsky service readme
* build
* tidy
* tidy, fix pds proxy tests
* fix
* fix bsky entrypoint deps
* support http2 grpc client
* build
* fix dataplane bad tls config/default
* support multiple dataplane urls, retry when unavailable
* build
* tidy/fix
* move dataplane mock tests into their own dir
* cover label hydration through timeline test
* bring back labels in appview tests
* remove unused db primary/replica/coordinator from bsky dataplane
* bsky proto add cids to contracts, buf codegen
* sync-up bsky data-plane w/ codegen updates
* start using dataplane interaction endpoints
* add file
* avoid overfetching from dataplane, plumb feed items and cids
* pass refs through for post viewer state
* Lexicons: Add labeler prefs, labeler declaration, and get labeler routes
* Add labelerViewBasic and update embed views
* Fix typo
* switch list feeds to use feed item in dataplane
* handle not found err on get-thread dataplane call
* support use of search service rather than dataplane methods
* mark some appview v2 todos
* tidy
* still use dataplane on search endpoints when search service is not configured
* fix pds test
* Switch to labelerViewDetailed
* Move label and report values to refs
* Add getActorLabelers
* lint
* fix up bsky tests & snaps
* small diff to open pr
* rm new line
* codegen schemas
* tidy migrations
* table + indexing
* protos
* rename lexicons
* views, hydration + rename lexicons
* rest of routes
* data plane routes
* parse labelers from req
* fix appview-v2 docker build
* Support label issuer tied to appview v2 (#2033)
support label issuer tied to appview
* hydrate context
* tidy header logic
* integrating into more routes
* more routes
* wrap up rest
* add mock labeler
* rework labelerlexicons
* tidy lexs
* codegen new lexicons
* integrate lexicon rework
* add proxy logic
* forward labeler headers through pds
* tweak label header parsing
* remove did from scheams
* update indexing for lexs
* tests for mod service views
* label hydration test
* Add 'associated' to profileViewDetailed
* Rename labelers to mods in preferences
* Change uri to did in mod preferences
* couple more
* syntax tweaks
* integrate updated lexicons
* update view snap
* handle mod service embeds
* tidy
* fix mock
* lint
* base default labels of config var
* fix label hydration
* Appview v2: handle empty cursor on list notifications (#2017)
handle empty cursor on appview listnotifs
* Update appview v2 to use author feed enum (#2047)
* update bsky protos with author feed enum, misc feed item changes
* support new author feed enums in dataplane
* fix build
* Appview v2: utilize sorted-at field in bsky protos (#2050)
utilize new sorted-at field in bsky protos
* remove all dataplane usage of GetLikeCounts, switch to GetInteractionCounts
* Appview v2, sync w/ changes to protos (#2071)
* sync bsky protos
* sync-up bsky implementation w/ proto changes
* Appview v2 initial implementation for getPopularFeedGenerators (#2072)
add an initial implementation for getPopularFeedGenerators on appview v2
* merge
* fixes
* fix feed tests
* fix bsync mock
* format
* remove unused config
* fix lockfile
* another lockfile fix
* fix duplicate type
* fix dupplicate test
* Appview v2 handling clearly bad cursors (#2092)
* make mock dataplane cursors different from v1 cursors
* fail open on clearly bad appview cursors
* fix pds appview proxy snaps
* Appview v2 no notifs seen behavior (#2096)
* alter behavior for presenting notifications w/ no last-seen time
* fix pds proxy tests
* Appview v2 dataplane retries based on client host (#2098)
choose dataplane client for retries based on host when possible/relevant
* don't apply negated labels
* display suspensions on actor profile in appview v2
* Appview v2 use dataplane for identity lookups (#2095)
* update bsky proto w/ identity methods
* setup identity endpoints on mock dataplane
* move from idresolver to dataplane for identity lookups on appview
* tidy
* Appview v2: apply safe takedown refs to records, actors (#2107)
apply safe takedown refs to records, actors
* Fix timing on appview v2 repo rev header (#2113)
fix timing on appview repo rev
* fix post thread responses
* Appview v2 don't apply 3p self blocks (#2112)
do not apply 3p self-blocks
* Appview v2 search for feed generators (#2118)
* add protos for feedgen search
* support feed search on getPopularFeedGenerators
* Appview v2 config tidy (#2117)
* remove mod and triage roles from appview
* rename cdn and search config
* remove custom feed harness from appview v2
* Appview v2: don't apply missing modlists (#2122)
* dont apply missing mod lists
* update mock dataplane
* Update packages/bsky/src/hydration/hydrator.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* refactor & document a bit better
* fix up other routes
---------
Co-authored-by: devin ivy <devinivy@gmail.com>
* Appview v2 enforce post thread root boundary (#2120)
* enforce post thread root boundary
* test thread root boundary
* Appview v2 fix admin environment variable (#2137)
fix admin env in appview v2
* Remove re-pagination from getSuggestions (#2145)
* remove re-pagination from getSuggestions
* fix test
* Adjust wording for account suspension (#2153)
adjust wording for account suspension
* Appview v2: fix not-found and blocked uris in threads (#2201)
* fix uris of not-found and blocked posts in threads
* update snaps
* ✨ Show author feed of takendown author to admins only (#2197)
* fold in cid, auth, tracing, node version changes
* remove dead config from bsky service entrypoint
* build
* remove ozone test codepaths for appview v2
* tidy, docs fix
* fix test
* add additional user counts
* add associated data to profiles
* update snaps
* update to is_mod_service
* format
* tidy
* 3p labeler sdk updates (#2176)
* Update sdk to support 3p labeler preferences
* Stick with intolerance instead of hate for the label group id
* wip expand labels and label groups
* Output moderationOpts (computed) and modsPref (unaltered)
* Add tests for enabling/disabling mod services
* Add atproto-labelers header config
* Expand labels and label groups in definitions
* Fix tests
* Tweaks to labels
* Remove label descriptions and improve output types on labels and label groups
* Add mocker to exported API
* Improve types of label and label group definitions
* Rework moderation prefs to continue using global labelgroup settings and only disable label groups per moderator
* Simplify encoding of the label preferences in definitions
* Add target constraints to labels
* Refactor the moderation sdk to derive more behaviors from the definition files
* Small cleanup
* Add hiding tool to modsdk
* Track filter causes
* Make mute state an alert
* Fix: dont blur profileview for blocks
* Prioritize causes by severity
* Add moderateNotification() and drop quote post moderation code
* Add mocker functions for notifications
* Improve mock data
* Lexicon: Add custom label definitions and remove modservice descriptions
* Lexicon: Update moderation prefs
* SDK updates: remove label groups, reduce builtin labels, update mod-preference apis
* Lexicon: Update global labels with new reduced set
* Lexicon: Remove moderation.getService and add detailed option to getServices
* Lexicons: add severity=none to custom label value defs
* Implement custom label-value definition tooling
* All custom labels are no-self
* Backend impl for labeler lexicon updates (#2258)
* codegen
* clean up impl
* fix up tests
* Lexicon: modservice -> labeler
* Remove x- prefix behavior; add label value syntax rules; add custom label precedence rules
* Lexicon: Remove the ability to choose a defaultSetting from custom labels
* Rework test suites
* Give behaviors to all labels regardless of target
* sync up backend with lex changes
* fix labelers in dev-env agent
* lint protos
* update protos & views
* small bugfix & update tests
* tweak protos
* fix build issue from merge
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
* add cid to ListView and ListViewBasic
* add test for mute list embed
* add support for list embeds views
* test
* port to appview
* update missing snap
---------
Co-authored-by: dholms <dtholmgren@gmail.com>
* actor type in profiles
* skeleton lex
* more schemas
* first pass at feed composition
* cleanup services
* more tidying
* fix up tests
* bookmarking schemas
* make feeds records instead of actors
* some cleanup
* fleshing out feeds as records
* wip
* working into cfg & dev-env
* fix appview test util
* bookmarking + views
* tidy
* add displayName
* db migration
* wip test
* blob processing
* first test working!
* Sync-up snapshots
* wip
* Fix pds ctx type
* Sync lexicons on pds and api
* wip
* fully refactor
* Prep for open union in reply data in bsky appview, sync-up snapshots
* msgs & ports
* Make pds did cache sqlite-friendly
* Start scaffolding more feedgen tests
* revamp appview tests
* revamp pds tests
* merge in dev env utils
* ensure plc close
* add feed gen to dev-env
* second feed gen
* named feeds in mock
* fix issue with dev-env pds appview
* update bsky snaps
* Flesh-out some feed gen tests
* Fix feed gen reply test
* Test feed bookmarking
* update skeleton defs
* merge
* Sync-up pds with updated feed skeleton contract
* Fix feed bookmark viewer state
* add feed embeds
* Test inaccurate feed gen repost
* Tighten some error behaviors around feed gens
* generator view in record embed
* test
* bookmarked/subscribed -> saved
* updated snaps
* name & descript on alices feed
* add like counts
* proposed lex
* tweak
* fetch feed generator details
* add describeFeedGen to dev-env
* adds tests for getFeedGenerator
* Update lexicons/app/bsky/feed/getFeedGenerator.json
Co-authored-by: devin ivy <devinivy@gmail.com>
* update describe
* add cid to feed gen view
* add alice feed gen avatar
* generator displayName required
* update snaps
* Fix dev env public url (#1058)
* Fix the public url used by dev-env
* move publicUrl location
---------
Co-authored-by: dholms <dtholmgren@gmail.com>
* add defs for feed gen prefs & decrease displayName length
* build branch
* updated how saved feeds works
* cleanup lex
* mount route
* Starter bsky algos (#1046)
* implement some starter bsky algos
* add team dids
* tidy
* add quotepost check to whats hot classic
* tests
* carve out feedgen section of pds
* apply limits
* add describeFeedGenerator route
* Whats hot new (#1060)
* map out new whats hot
* only available in postgres
* Fix dev env public url (#1058)
* Fix the public url used by dev-env
* move publicUrl location
---------
Co-authored-by: dholms <dtholmgren@gmail.com>
* add defs for feed gen prefs & decrease displayName length
* build branch
* updated how saved feeds works
* build algos branch
* remove relative imports to @atproto/api
* update hot classic name
* rm build
---------
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
* v0.3.6
* v0.3.7
* v0.1.10
* update name of skyline
* build branch
* aws build
* getFeedGenerators test
* For now only use new replyRef union for custom feeds, for compat
* Don't use replyRef union on new feeds yet
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
* block lexicons
* table schema
* block processing
* Actor-block migration
* Add blocked post lexicon to getPostThread
* getBlocks routes
* Add missing lexicons to bsky
* Add block state to actor views
* throw on blocked accounts in getAuthorFeed
* blcok state in profile views
* Update getPostThread to support block views, update existing test
* strip out blocks out oof timeline & popular
* more routes
* Test blocked posts in threads
* Support blocks on record embeds
* notifs
* Update pds test snapshots w/ block state
* fix search
* Update pds actor search snapshots w/ blocks
* test
* tests
* update lex
* fix up from returning blocking as record
* ci for branch
* fix build issue in bsky
* more blocks tests
* fix actor search test
* hook getBlocks up to router
* bugfix & actually test lol
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* remove return in test
* couple of fixups in other pacakges
* Add dummy checks to declaration and follow app migrations, remove paranoid join
* update db nsid migration
* Ensure there are writes in follow app migration
* Add dumy check to votes-to-likes app migration, tidy
* Ensure there are writes in vote-to-like app migration
* update migration name
* pr feedback
* count utf8 & grapheme length
* add maxUtf8
* siwtch max semantics
* plural
* update post schema
* added bytes & cid refs
* add ipld<>json
* fixin up a could tings
* Add app.bsky.richtext.facet, replace post entities with facets
* plural actors
* wip
* Setup backlinks table on pds
* wip
* send & recieve cids/bytes with xrpc
* Track backlinks when indexing records on pds
* handle ipld vals in xrpc server
* added cids & bytes to codegen
* In createRecord, add deletions to avoid duplicate likes/follows/reposts
* Tests and fixes for prevention of dupe follows, likes, reposts
* Backlink migration tidy
* cleanup dag json parser
* Fix dupe backlink inserts
* Tidy
* blob refs + codegen
* Make profile displayName optional
* Test view and updateProfile for empty display name
* working into pds
* Make aggregate counts optional on post and profile views
* Make viewer state optional on post view for consistency
* Remove deprecated myState field on profile view
* Tidy repo method descriptions
* tests & types & fixes
* Implementation and tests for putRecord
* Remove updateProfile method
* Update repo service so that head can be taken for update externally
* Lex updates for compare-and-swap records/commits
* Add error to lex for bad repo compare-and-swaps
* Improve update-at-head thru repo service
* common package
* Implement and test compare-and-swaps on repo write methods
* Use lex discriminator for applyWrites
* Remove post entity/facet index
* Update lex descriptions to clarify repo write semantics
* Make deleteRecord idempotent w/ tests
* cleanup
* fix things up
* adding more formats
* tests
* updating schema
* Only generate tid rkeys on pds, support literal rkeys on client
* Add backlink indexes
* Update format of post embed views, fix external uri validation
* fixing up tests
* Include embeds on record embeds
* cleanup
* Notify users when they are quoted
* Remove determineRkey indirection
* fix api tests
* support concatenated cbor
* integrating to server
* re-enable tests
* fix up tests
* Thread compare-and-swaps down into repo service rather than use pinned storage
* Tidy
* Update packages/common/tests/ipld-multi.test.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* Update packages/lexicon/src/validators/formats.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* pr feedback
* pr feedback
* Add postgres-specific migration path for missing profile display names
* Tidy/clarify deep embeds
* Tidy
* rm unused escape
* decrease crud race count
* update subscribeRepos lexicon
* Fix applyWrite lexicon re: collection fields
* sign post event type
* update cids & bytes json encoding
* update lex blob & cid-link types
* updated codegen & pds
* number -> float
* missed a couple
* remove old image constraints
* pr feedback + descripts
* no hardcoded port numbers
* remove separate tooLarge evt
* fix dumb build error
* fixin gup lex + xrpc server
* better parsing of message types
* dont mutate body in subscription
* bugfix in subscription
* rm commented out code
* init feature branch
* undo
* Remove old lexicons
* Remove creator from profile view
* wip
* rework seqs
* fixed up tests
* bug fixing
* sequence handles & notify in dbTxn
* tidy
* update lex to include times
* test syncing handle changes
* one more fix
* handle too big evts
* dont thread sequencer through everything
* Split common into server vs web-friendly versions
* Make lexicon, identifier web-safe using common-web
* Switch api package to be a browser build, fix identifier package for browser bundling
* Fix pds and repo for lexicon package changes, tidy
* Make common-web a browser build, tidy
* fixing up deps
* fix up test
* turn off caching in actions
* Standardize repo write interfaces around repo input
* Update repo write endpoints for repo input field
* Remove scene follows during app migration
* API package updates (#712)
* Add bsky agent and various sugars to the api package
* Add richtext library to api package
* Update richtext to use facets and deprecate entities
* Update richtext to use utf8 indices
* Richtext converts deprecated entity indices from utf16 locations to utf8 locations
* Add note about encodings in the lexicon
* Add RichText facet detection
* Remove dead code
* Add deprecation notices to lexicons
* Usability improvements to RichText
* Update the api package readme
* Add RichText#detectFacetsWithoutResolution
* Add upsertProfile to bsky-agent
* Update packages/pds/src/api/com/atproto/repo/applyWrites.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* pr feedback
* fix flaky timing streaming tests
* simplify emptyPromise
* fixed up open handles
* fix missed repo syntax
* fix error in test from fkey constraint
* fix another api agent bug
* Embed consistency, add complex record embed
* Tidy embed lex descriptions
* rename pg schemas
* use swc for jest
* fix up deps
* cleanup
* Update pds indexing, views, tests for complex record embeds
* fixing up profile view semantics
* wip
* update snaps
* Rename embed.complexRecord to embed.recordWithMedia
* Tidy aroud record w/ media embeds
* Add grapheme utilities to api RichText (#720)
Co-authored-by: dholms <dtholmgren@gmail.com>
* Fix: app.bsky.feed.getPostThread#... to app.bsky.feed.defs#... (#726)
* Update bskyagent to use repo param
* Minor typing fix
* Add exports to api package: blobref & lex/json converters (#727)
* Add exports to api package: BlobRef & lex/json converters
* Add an example react-native fetch handler
* Switch all lingering references of recordRef to strongRef
* Update lexicon for richtext facets to have multiple features, byte slice rather than text slice
* Implement multi-feature richtext facets on pds
* Update api package to use updated richtext facets
* Minor fixes to admin repo/record views
* Fix app migration exports, remove old app migration
* Fix: sort richtext facets so they can render correctly
* Disable app migration dummy checks that don't work on live deploy
* Optimize lex de/serialization using simple checks
* Tidy comment typos
* App migration to cleanup notifications for likes, follows, old scene notifs
* Fix notification reason for change from vote to like
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
* Add lexicon for post embeds
* Model post embeds
* Handle and present post embeds
* Test and tidy post embeds
* Generalize post embeds to record embeds
* Tidy
devin ivy
Hailey
Eric Bailey
Daniel Holmgren
bnewbold
Emily Liu
Ansh