Daniel Holmgren
265ce86e15
Ozone chat integration ( #2490 )
...
* wip
* schema tweaks
* room -> chat
* missed one
* missed another
* deleted messages + incoming message settings
* add schemas for blocking users
* chat update schemas
* make updateUserSettings more generic
* fix dangling ref
* fix dangling refs
* mute chats & leave chats
* schema consistency, naming
* lexicon for updating chat read state
* dm lexicon updates, add batch send endpoint
* dm sender on deleted message
* Add codegen for temp lex
* lexicon: switch temp.dm to chat.bsky namespace, misc fixes
* lexicon: codegen
* lexicon: fix some chat/convo terminology
* lexicon: codegen
* moderator schemas
* codegen
* lexicon: consistency in chat lexicons
* lexicon: add chat to associated profile info
* add ref to updateActorAccess
* add annotation on convoView for disabled members
* codegen
* ozone lexicons
* scaffold routes
* hook up chat moderation routes to ozone
* add message subject for reports/events
* remove dm references from ozone schemas
* ensure report exists before serving getMessageContext
* 🐛 Fix typo in column name
* adjust before/after instead of throwing
* add dev script
* add dev-env introspection server
* hook up chat mod routes to pds & ozone
* ✨ Add message ref subject in eventView def
* fix typo
* add chat reporting test
* fix test
* update dev-env chat did
* update dms server routes name
* pr feedback
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
2024-05-15 18:52:47 -05:00
Daniel Holmgren
f65de89eed
Feature: Appview v2 ( #1924 )
...
* add buf & connectrpc, codegen client
* lint
* prettier ignore
* fix prettier ignore
* tidy & add tests
* filler commit
* rm filler
* server boilerplate
* follows impl
* posts impl
* posts & likes impl
* repost impl
* profiles & handle null values
* list impl
* mutes impl
* blocks impl
* misc
* feed gen impl
* label impl
* notifs impl
* feeds impl
* threads impl
* early sketchwork
* wip
* stub out thick client
* in-progress work on hydrator
* tweak
* hydrate profile labels, detail lists
* feedgen hydration
* protobuf tweaks
* more protobuf tweaks
* wip
* snake case
* moar snake case
* tidy actor hydration
* tidy parsing
* type fixes, renaming, comments in hydrator
* hydrate list items and likes
* hydrate notifications
* feed hydration
* graph & label hydration
* more record protobufs
* pluralize
* tweak pbs
* use new methods
* Setup dataplane grpc client/mock server (#1921 )
* add buf & connectrpc, codegen client
* lint
* prettier ignore
* fix prettier ignore
* tidy & add tests
* add record getter mocks
* post view hydration
* fix up mock dataplane to match new protos
* missed one
* wire up dataplane in ctx & dev-env
* adding some basic views
* feed hydration, add blocks to post hydration
* pass over notification hydration
* tidy
* merge
* implement getProfile
* hydrate post aggregation and viewer state
* fix
* fix codegen
* get some tests passing!
* add takedowns & some like bugfixing
* all profile tests passing!
* likes test
* follow endpoints using data plane
* reorg follow block rules
* reposts
* post views!
* implement getList w/ dataplane caveat
* adjust dataplane getListMembers to return listitem uris
* implement getListMutes and -Blocks w/ dataplane
* suggestions
* timeline
* misc view fixes
* view fixes for mutes, self-mute/block
* author feed
* feed gen routes
* tidy
* misc block/mute fixes
* list feed & actor likes
* implement getLists, fix some empty cursors
* implement getMutes, empty profile description fix
* implement getBlocks, block application fix
* implement getSuggestedFollowsByActor, needs some fixes
* feed generation
* search routes
* threads
* tidy
* fix some snaps
* fix getSuggestedFollowsByActor
* implement listNotifications
* implement getUnreadCount w/ dataplane
* implement notifications.updateSeen w/ dataplane
* 3rd party blocking tests
* blocked profile viewer
* add search mocks
* refactor getFeed
* createPipeline -> createPipelineNew
* basic replygating functionality on dataplane w/o filtering violating replies
* hack threadgates into dataplane, apply gates
* deterministic thread orders in dataplane
* misc cleanup around dataplane
* upgrade typescript to v5.3
* update typescript linter deps
* sync bsky proto, codegen
* update dataplane, sync with bsky proto updates
* remove indexer, ingester, daemon, moderation services from appview
* convert more bsky internals to dataplane, remove custom feedgens, implement mute/unmuting in mock dataplane
* remove bsky services. db and indexing logic into mock dataplane.
* remove tests not needed by appview v2, misc reorg
* add basic in-mem repo subscription to dataplane mock
* fix dev-env, bsky tests, bsky build
* cull bsky service entrypoint
* add bsky service readme
* build
* tidy
* tidy, fix pds proxy tests
* fix
* fix bsky entrypoint deps
* support http2 grpc client
* build
* fix dataplane bad tls config/default
* support multiple dataplane urls, retry when unavailable
* build
* tidy/fix
* move dataplane mock tests into their own dir
* cover label hydration through timeline test
* bring back labels in appview tests
* remove unused db primary/replica/coordinator from bsky dataplane
* bsky proto add cids to contracts, buf codegen
* sync-up bsky data-plane w/ codegen updates
* start using dataplane interaction endpoints
* add file
* avoid overfetching from dataplane, plumb feed items and cids
* pass refs through for post viewer state
* switch list feeds to use feed item in dataplane
* handle not found err on get-thread dataplane call
* support use of search service rather than dataplane methods
* mark some appview v2 todos
* tidy
* still use dataplane on search endpoints when search service is not configured
* fix pds test
* fix up bsky tests & snaps
* tidy migrations
* fix appview-v2 docker build
* Support label issuer tied to appview v2 (#2033 )
support label issuer tied to appview
* Appview v2: handle empty cursor on list notifications (#2017 )
handle empty cursor on appview listnotifs
* Update appview v2 to use author feed enum (#2047 )
* update bsky protos with author feed enum, misc feed item changes
* support new author feed enums in dataplane
* fix build
* Appview v2: utilize sorted-at field in bsky protos (#2050 )
utilize new sorted-at field in bsky protos
* remove all dataplane usage of GetLikeCounts, switch to GetInteractionCounts
* Appview v2, sync w/ changes to protos (#2071 )
* sync bsky protos
* sync-up bsky implementation w/ proto changes
* Appview v2 initial implementation for getPopularFeedGenerators (#2072 )
add an initial implementation for getPopularFeedGenerators on appview v2
* merge
* fixes
* fix feed tests
* fix bsync mock
* format
* remove unused config
* fix lockfile
* another lockfile fix
* fix duplicate type
* fix dupplicate test
* Appview v2 handling clearly bad cursors (#2092 )
* make mock dataplane cursors different from v1 cursors
* fail open on clearly bad appview cursors
* fix pds appview proxy snaps
* Appview v2 no notifs seen behavior (#2096 )
* alter behavior for presenting notifications w/ no last-seen time
* fix pds proxy tests
* Appview v2 dataplane retries based on client host (#2098 )
choose dataplane client for retries based on host when possible/relevant
* don't apply negated labels
* display suspensions on actor profile in appview v2
* Appview v2 use dataplane for identity lookups (#2095 )
* update bsky proto w/ identity methods
* setup identity endpoints on mock dataplane
* move from idresolver to dataplane for identity lookups on appview
* tidy
* Appview v2: apply safe takedown refs to records, actors (#2107 )
apply safe takedown refs to records, actors
* Fix timing on appview v2 repo rev header (#2113 )
fix timing on appview repo rev
* fix post thread responses
* Appview v2 don't apply 3p self blocks (#2112 )
do not apply 3p self-blocks
* Appview v2 search for feed generators (#2118 )
* add protos for feedgen search
* support feed search on getPopularFeedGenerators
* Appview v2 config tidy (#2117 )
* remove mod and triage roles from appview
* rename cdn and search config
* remove custom feed harness from appview v2
* Appview v2: don't apply missing modlists (#2122 )
* dont apply missing mod lists
* update mock dataplane
* Update packages/bsky/src/hydration/hydrator.ts
Co-authored-by: devin ivy <devinivy@gmail.com>
* refactor & document a bit better
* fix up other routes
---------
Co-authored-by: devin ivy <devinivy@gmail.com>
* Appview v2 enforce post thread root boundary (#2120 )
* enforce post thread root boundary
* test thread root boundary
* Appview v2 fix admin environment variable (#2137 )
fix admin env in appview v2
* Remove re-pagination from getSuggestions (#2145 )
* remove re-pagination from getSuggestions
* fix test
* Adjust wording for account suspension (#2153 )
adjust wording for account suspension
* Appview v2: fix not-found and blocked uris in threads (#2201 )
* fix uris of not-found and blocked posts in threads
* update snaps
* ✨ Show author feed of takendown author to admins only (#2197 )
* fold in cid, auth, tracing, node version changes
* remove dead config from bsky service entrypoint
* build
* remove ozone test codepaths for appview v2
* tidy, docs fix
---------
Co-authored-by: Devin Ivy <devinivy@gmail.com>
Co-authored-by: Foysal Ahamed <foysal@blueskyweb.xyz>
2024-02-27 15:22:55 -05:00
Daniel Holmgren
de2dbc2903
Split out moderation backend ( #1970 )
...
* mv appview
* copy
* finalize copy
* package names
* big WIP
* first pass at mod servce
* some tidy
* tidy & fix compiler errors
* rename to ozone, db migrations, add to dev-env & pds cfg
* getRecord & getRepo mostly working
* fix open handle
* get record tests all working
* moderation events working
* statuses working
* tidy test suite
* search repos
* server & db tests
* moderation tests
* wip daemon + push events
* pds fanout working
* fix db test
* fanning takedowns out to appview
* rm try/catch
* bsky moderation test
* introduce mod subject wrappers
* more tidy
* refactor event reversal
* tidy some db stuff
* tidy
* rename service to mod-service
* fix test
* tidy config
* refactor auth in bsky
* wip patching up auto-mod
* add label ingester in appview
* fix a couple build issues
* fix some timing bugs
* tidy polling logic
* fix up tests
* fix some pds tests
* eslint ignore
* fix ozone tests
* move seeds to dev-env
* move images around
* fix db schemas
* use service auth admin reqs
* fix remaining tests
* auth tests bsky
* another test
* random tidy
* fix up search
* clean up bsky mod service
* more tidy
* default attempts to 0
* tidy old test
* random tidy
* tidy package.json
* tidy logger
* takedownId -> takedownRef
* misc pr feedback
* split daemon out from ozone application
* fix blob takedown mgiration
* refactor ozone config
* do push event fanout on write instead of on read
* make suspend error work again
* add attempts check & add supporting index
* fix takedown test ref
* get tests working
* rm old test
* fix timing bug in event pusher tests
* attempt another fix for timing bug
* await req
* service files
* remove labelerDid cfg
* update snaps for labeler did + some cfg changes
* fix more snaps
* pnpm i
* build ozone images
* build
* make label provider optional
* fix build issues
* fix build
* fix build
* build pds
* build on ghcr
* fix syntax in entry
* another fix
* use correct import
* export logger
* remove event reverser
* adjust push event fanout
* push out multiple
* remove builds
2024-01-05 17:06:54 -06:00
Foysal Ahamed
devin ivy
Daniel Holmgren