* adding to moderation flows
* some bugfixing in labels
* hardcode profile nsid
* make labels off moderation action
* db updates
* wip
* report labels in admin views & reverse moderation actions
* Test admin get record and repo w/ labels
* update db
* fix
* exclude negs from labels
* exclude neg on moderation views as well
* Check-in missing lex
* Check-in missing lex
* In-progress admin label tests
* Test label creation/reversal via actions
* Admin label test snapshots (#808)
* new snapshots for label on user
* fix get moderation action snap
* fix dev-env
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* com.atproto.label.label lexicon
For persisting un-signed label objects in repositories.
There will likely be a handful of XRPC query endpoints in the
com.atproto.label namespace in the near future.
* lexicons: com.atproto.label.subscribeLabels
* lexicons: com.atproto.label.query
* lexicons: label updates post lex-refactor
* lexicons: rename label.query to label.queryLabels
* lexicons: queryLabels larger batch sizes; improve description
* lexicons: add labels to profiles, posts, notifications
* lexicons: add negation ('neg') flag to labels
* lexicon: label vals on moderation actions
This is a mechanism to create (or negate) labels as part of a moderation
action. The fields are optional, so this is backwards compatible, and if
we wanted to embed entire labels in the future (instead of just "val"
strings), that would be possible using a different field name.
* lexicons: update label.label to label.defs#label
* codegen
---------
Co-authored-by: dholms <dtholmgren@gmail.com>
* filter embeds from get popular
* Truncate profile info to satisfy validation (#746)
* Remove trailing replacement character when utf8-truncating (#747)
* Truncate profile info to satisfy validation
* Fix utf8 truncation w/ replacement character
* filter replies
* delete embeds from records on getPopular
* Update profile display name and description lengths to be based on graphemes (#748)
* @atproto api v0.2.1
* Tidy
---------
Co-authored-by: devin ivy <devinivy@gmail.com>
* remove return in test
* couple of fixups in other pacakges
* Add dummy checks to declaration and follow app migrations, remove paranoid join
* update db nsid migration
* Ensure there are writes in follow app migration
* Add dumy check to votes-to-likes app migration, tidy
* Ensure there are writes in vote-to-like app migration
* update migration name
* pr feedback
* count utf8 & grapheme length
* add maxUtf8
* siwtch max semantics
* plural
* update post schema
* added bytes & cid refs
* add ipld<>json
* fixin up a could tings
* Add app.bsky.richtext.facet, replace post entities with facets
* plural actors
* wip
* Setup backlinks table on pds
* wip
* send & recieve cids/bytes with xrpc
* Track backlinks when indexing records on pds
* handle ipld vals in xrpc server
* added cids & bytes to codegen
* In createRecord, add deletions to avoid duplicate likes/follows/reposts
* Tests and fixes for prevention of dupe follows, likes, reposts
* Backlink migration tidy
* cleanup dag json parser
* Fix dupe backlink inserts
* Tidy
* blob refs + codegen
* Make profile displayName optional
* Test view and updateProfile for empty display name
* working into pds
* Make aggregate counts optional on post and profile views
* Make viewer state optional on post view for consistency
* Remove deprecated myState field on profile view
* Tidy repo method descriptions
* tests & types & fixes
* Implementation and tests for putRecord
* Remove updateProfile method
* Update repo service so that head can be taken for update externally
* Lex updates for compare-and-swap records/commits
* Add error to lex for bad repo compare-and-swaps
* Improve update-at-head thru repo service
* common package
* Implement and test compare-and-swaps on repo write methods
* Use lex discriminator for applyWrites
* Remove post entity/facet index
* Update lex descriptions to clarify repo write semantics
* Make deleteRecord idempotent w/ tests
* cleanup
* fix things up
* adding more formats
* tests
* updating schema
* Only generate tid rkeys on pds, support literal rkeys on client
* Add backlink indexes
* Update format of post embed views, fix external uri validation
* fixing up tests
* Include embeds on record embeds
* cleanup
* Notify users when they are quoted
* Remove determineRkey indirection
* fix api tests
* support concatenated cbor
* integrating to server
* re-enable tests
* fix up tests
* Thread compare-and-swaps down into repo service rather than use pinned storage
* Tidy
* Update packages/common/tests/ipld-multi.test.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* Update packages/lexicon/src/validators/formats.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* pr feedback
* pr feedback
* Add postgres-specific migration path for missing profile display names
* Tidy/clarify deep embeds
* Tidy
* rm unused escape
* decrease crud race count
* update subscribeRepos lexicon
* Fix applyWrite lexicon re: collection fields
* sign post event type
* update cids & bytes json encoding
* update lex blob & cid-link types
* updated codegen & pds
* number -> float
* missed a couple
* remove old image constraints
* pr feedback + descripts
* no hardcoded port numbers
* remove separate tooLarge evt
* fix dumb build error
* fixin gup lex + xrpc server
* better parsing of message types
* dont mutate body in subscription
* bugfix in subscription
* rm commented out code
* init feature branch
* undo
* Remove old lexicons
* Remove creator from profile view
* wip
* rework seqs
* fixed up tests
* bug fixing
* sequence handles & notify in dbTxn
* tidy
* update lex to include times
* test syncing handle changes
* one more fix
* handle too big evts
* dont thread sequencer through everything
* Split common into server vs web-friendly versions
* Make lexicon, identifier web-safe using common-web
* Switch api package to be a browser build, fix identifier package for browser bundling
* Fix pds and repo for lexicon package changes, tidy
* Make common-web a browser build, tidy
* fixing up deps
* fix up test
* turn off caching in actions
* Standardize repo write interfaces around repo input
* Update repo write endpoints for repo input field
* Remove scene follows during app migration
* API package updates (#712)
* Add bsky agent and various sugars to the api package
* Add richtext library to api package
* Update richtext to use facets and deprecate entities
* Update richtext to use utf8 indices
* Richtext converts deprecated entity indices from utf16 locations to utf8 locations
* Add note about encodings in the lexicon
* Add RichText facet detection
* Remove dead code
* Add deprecation notices to lexicons
* Usability improvements to RichText
* Update the api package readme
* Add RichText#detectFacetsWithoutResolution
* Add upsertProfile to bsky-agent
* Update packages/pds/src/api/com/atproto/repo/applyWrites.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* pr feedback
* fix flaky timing streaming tests
* simplify emptyPromise
* fixed up open handles
* fix missed repo syntax
* fix error in test from fkey constraint
* fix another api agent bug
* Embed consistency, add complex record embed
* Tidy embed lex descriptions
* rename pg schemas
* use swc for jest
* fix up deps
* cleanup
* Update pds indexing, views, tests for complex record embeds
* fixing up profile view semantics
* wip
* update snaps
* Rename embed.complexRecord to embed.recordWithMedia
* Tidy aroud record w/ media embeds
* Add grapheme utilities to api RichText (#720)
Co-authored-by: dholms <dtholmgren@gmail.com>
* Fix: app.bsky.feed.getPostThread#... to app.bsky.feed.defs#... (#726)
* Update bskyagent to use repo param
* Minor typing fix
* Add exports to api package: blobref & lex/json converters (#727)
* Add exports to api package: BlobRef & lex/json converters
* Add an example react-native fetch handler
* Switch all lingering references of recordRef to strongRef
* Update lexicon for richtext facets to have multiple features, byte slice rather than text slice
* Implement multi-feature richtext facets on pds
* Update api package to use updated richtext facets
* Minor fixes to admin repo/record views
* Fix app migration exports, remove old app migration
* Fix: sort richtext facets so they can render correctly
* Disable app migration dummy checks that don't work on live deploy
* Optimize lex de/serialization using simple checks
* Tidy comment typos
* App migration to cleanup notifications for likes, follows, old scene notifs
* Fix notification reason for change from vote to like
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Paul Frazee <pfrazee@gmail.com>
* Add lexicon for post embeds
* Model post embeds
* Handle and present post embeds
* Test and tidy post embeds
* Generalize post embeds to record embeds
* Tidy
* Add lexicon for getProfiles
* Split out presentation of profiles, implement getProfiles, test
* Cull actor/profile view lexicons down to three variants
* Map viewer in profile view, mark todos, lex tweak
* Apply basic profile view to methods, lex fix
* Switch searchTypeahead to actor-with-info view
* Switch getVotes to actor-with-info view
* Switch getMutes to actor-with-info view
* Switch getFollows and getFollowers to actor-with-info view
* Switch getRepostedBy to actor-with-info view
* Switch feed and notification views to use updated actor-with-info view
* change pkey on user table to did & rename to user_account
* migration
* tidy
* fixes suggested by bryn
* missed merge thing
* some updating hanldes
scheams
* impl + passing test
* more handle tests
* tidy
* update did doc + some new tests
* one more test
* test handle casing
* Fix did pkey down migration
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* Scaffold xrpc-stream
* Write and test stream frames
* Write and test stream server
* Test bad stream frame parsing cases
* Proof of concept streaming on xrpc-server
* Test bad streaming endpoint
* Make frame stream to fix buffering frame bytes together
* reorg
* some lex-cli things
* better codegen for subscriptions
* scaffolding repo subscriptions
* wip
* remove repo ops
* setup notify/listen for db
* end pool to fix hanging test
* small comment in test
* basic sequencer
* some refactoring
* switch to event emitter
* reconnect on listener error
* rename notifyClient
* remove payload on channels
* pr feedback
* subscribeRepo outbox
* some cleanup
* wip
* wip
* bugfixin
* only send msgs after tx is committed
* better handle event-emitter -> generator
* max buffer size
* cleanup
* Sync-up xrpc subscriptions with current iteration of the spec
* Allow missing encoding for subscriptions
* track sequencedAt & eventType as well
* Resolve streaming type codes, streaming tests
* Fix interface for hooking into http server for streaming, fix stream routing
* Minor reorg how streaming is enabled in xrpc-server
* Server codegen for xrpc subscriptions, supporting types in xrpc-server
* fix up buffer overloading
* Rename data frame to message frame, rename subscription lex output to message
* Move body param first for streaming frames
* Tidy
* clean up evt types
* buff up tests
* missed merge conflict
* new schema
* blobs on subscriptions
* rm genned client subscription methods
* backfill limits
* testing subscription route & quick outbox bugfix
* fix up migration
* cascade on delete
* comments & naming
* fix dev env
* delete seqs on account deletion
* tidy
* fixing things up with db notify system for schemas
* fix duplicates in outbox
* tidy
* Break out of stream handler when socket closes
* fixing up some timing issues
* tidy
* terminate ws after using
* bump up timer on async reader
* fixing up NOTIFY in txns
* pr feedback
* pr bugfixes
* make order asc explicit
* bringing tests up to speed w atpagent
* bump up max listeners on sequencer
* increase timeouts for reading from generators
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
* Lexicon updates for moderating blobs
* Add tracking of blob takedowns to moderation system
* Handle blobs in record takedown reversal
* Apply blob takedowns w/ image invalidation
* Test blob takedowns
* Implement cloudfront image invalidator
* Update lexicons for current moderation action on records and blobs
* Fix migration for postgres
* Enforce single current action per repo and blob, present current action on repo and blob
* Test single active mod action per repo, record, and blob
* Continue to resolve handles for moderated repos
* Model mod actions for actor takedowns
* Add administration lex schemas for moderation actions
* Initial implementation of takedown mod action and reversal
* Refactor admin actions into a service
* Check for account takedown when creating new session or refreshing session
* Sweep usage of getUser() and getUserByEmail() for takedowns
* Begin sweep for account takedowns
* Tidy takedown checks
* Handle post author takedowns
* Fix tests and migrations for mod takedown
* Fix xrpc-server delayed read of blob payloads
* Test takedown mod action, misc fixes
* Remodel lex schemas for admin moderation actions
* Remove delcaration cid from actor takedowns, lex schema tidy
* Add notes
* Move moderation methods to com.atproto, based around repo rather than actor
* Moderation reports of repos and records
* Implement moderation report resolution method
* Tidy
* Place moderation-init migration after mute-init
* Allow services to reference each other
* Model record takedowns in db
* Lex schemas for record takedowns
* Service/methods for record takedowns, fix report view subject
* Apply record takedown to posts in feeds
* Test record takedown effects, fix reversal
* Support server moderation flag and acknowledge actions, fix action/report resolution for records
* Track repo takedowns on repo_root rather than did_handle
* Move repo/record reporting under com.atproto.report
* Colocate all moderation functionality into moderation service, combine from admin and repo services
* Tidy
* Tidy
* Unify mod report and action methods with new repo/record ref schemas
* Schemas for admin moderation views
* Tidy/codegen
* Fix test snapshot
* Break pds admin methods into separate files
* Implment admin repo search method
* Reorganize existing moderation views into the ModerationViews
* Lex schemas for admin get repo/record, implement get repo
* Implement admin get record view
* Tidy/codegen
* Tidy, fix snapshot
* Tidy, swap filenames
* Implement admin get moderation action view
* Implement admin get moderation report view
* Implement and test admin views for mod actions and reports
* Dry-up some admin and moderation-related seeds in the tests
* Fix xrpc fetch usage for newly required duplex option
* Admin view of repo and record details (#463)
* Reorganize existing moderation views into the ModerationViews
* Lex schemas for admin get repo/record, implement get repo
* Implement admin get record view
* Tidy/codegen
* Tidy, fix snapshot
* Model mod actions for actor takedowns
* Add administration lex schemas for moderation actions
* Initial implementation of takedown mod action and reversal
* Refactor admin actions into a service
* Check for account takedown when creating new session or refreshing session
* Sweep usage of getUser() and getUserByEmail() for takedowns
* Begin sweep for account takedowns
* Tidy takedown checks
* Handle post author takedowns
* Fix tests and migrations for mod takedown
* Fix xrpc-server delayed read of blob payloads
* Test takedown mod action, misc fixes
* Remodel lex schemas for admin moderation actions
* Remove delcaration cid from actor takedowns, lex schema tidy
* Add notes
* Move moderation methods to com.atproto, based around repo rather than actor
* Moderation reports of repos and records
* Implement moderation report resolution method
* Tidy
* Place moderation-init migration after mute-init
* Allow services to reference each other
* Model record takedowns in db
* Lex schemas for record takedowns
* Service/methods for record takedowns, fix report view subject
* Apply record takedown to posts in feeds
* Test record takedown effects, fix reversal
* Support server moderation flag and acknowledge actions, fix action/report resolution for records
* Track repo takedowns on repo_root rather than did_handle
* Move repo/record reporting under com.atproto.report
* Colocate all moderation functionality into moderation service, combine from admin and repo services
* Tidy
* Tidy
* Unify mod report and action methods with new repo/record ref schemas
* Procedures and view for user muting
* Update profile and feed/thread methods for muting, update tests
* Tidy
* Test muting for timeline, actor feed, thread, profile views, misc fixes
* Test mute views, misc fixes
* Add missing file
* Add per-user notification index
* Omit mention and reply notifications from muted users
* Test timing leeway
