615 Commits

Author SHA1 Message Date
Eric Bailey
ff0d2df6e3 use getActor, failing atm 2023-08-17 13:32:19 -05:00
Eric Bailey
1a5b422718 only include media posts by post creator 2023-08-17 12:10:36 -05:00
Foysal Ahamed
7c88b43fc5
Expose takendown profile, their follows and followers to mods (#1456)
*  Allow moderators to see takendown profiles

*  Allow moderators to see follows and followers of takendown account

* ♻️ Let auth check fail on optional verifier

* ♻️ Use role type to check moderator access
2023-08-17 09:47:34 -04:00
Daniel Holmgren
4241ee16ec
Handle revalidation (#1474)
* easier hanlde revalidation

* remove duplicate line

* backup handle nameservers on appview

* fix tests & add a couple
2023-08-16 18:34:19 -05:00
devin ivy
34b84131be
Handle db pool errors on appview (#1483)
handle db pool errors on bav
2023-08-16 18:22:42 -04:00
devin ivy
8eb930382c
Handle db client errors on appview (#1481)
handle db client errors on bav
2023-08-16 14:02:42 -04:00
Foysal Ahamed
7bc701e2e1
Allow creating moderation action with a duration (#1431)
*  Add action duration to action model

*  Add periodic moderation action reversal

*  Use actionDurationInHours and remove takedownExpiresAt

*  Destroy mod action reversal job

*  Add test for automatic moderation action reversal

* 🧹 Port over actionDuration to bsky package

* ♻️ Better naming for action duration and minor fixes

* ♻️ Optionally build expiresAt based on createdAt when logging action

* ♻️ Adjust constructor params

*  Copy over migration from pds to bsky package

* 🚨 Fix indentation

*  Propagate action reversal from app-view to pds

*  Revert locally only if pushAgent is not found

* build

* merge dbcoordinator work into periodic mod actions

* fix entrypoint for periodic mod reversals

* do not run periodic reversals on sequencer leader

* fix duration in action detail on bsky

* update env for pushing mod reversals

* allow zero-duration actions for testing

* remove build

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-08-15 15:44:53 -04:00
Eric Bailey
e0e7b39b16 add block and mute tests 2023-08-15 13:46:58 -05:00
Eric Bailey
79dcad4cf7 handle mutes 2023-08-15 11:25:38 -05:00
Eric Bailey
d9849b6776 get it working 2023-08-14 18:31:15 -05:00
Daniel Holmgren
73f64009a2
Appview database replicas (#1459)
* setup bsky so that writes always go to a db primary

* test bsky primary db functionality

* sprinkle db primary over moderation and view maintainer usage

* make bsky tests more strict on primary/secondary db, fixes

* maintain prev db config on appview, require primary db, make primary/replica explicit in config

* tidy

* sketch out db replicas

* wip

* support multiple tags per bsky replica, "any" tag, fallbacks and warns

* clarify tags

* use new coordinator api across routes & services

* wire-up replica tagging config

* tidy

* cleanup entroypoints and tests for db coordinator

* fix pds test

* re-hookup migrate db

* standardize env vars for db on bsky services

* tidy

* build

* fix bav api entrypoint

* support DB_POSTGRES_URL on ingester/indexer for backwards compat

* remove build

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-08-14 18:38:00 -04:00
Eric Bailey
b05986f7d1 codegen 2023-08-14 15:39:48 -05:00
Eric Bailey
c16fcb84bc v0.0.3 2023-08-14 10:01:21 -05:00
Daniel Holmgren
244bf46e74
Include limited info on blocked embeds (#1463)
* add block info to embeds

* fix codegen

* Correctly handle blocked embeds and add block-other cause

* update snaps

* Correctly identify blocking behavior in embeds

---------

Co-authored-by: Paul Frazee <pfrazee@gmail.com>
2023-08-10 15:54:24 -05:00
Daniel Holmgren
ba979b4cdb
Include limited viewer info on blocked post (#1457)
* include limited viewer info on block

* blockedAuthor schema

* make author required

* merge & update codegen

* patch up tests
2023-08-10 14:31:45 -05:00
Daniel Holmgren
3f0e1b2563
Read after write (#1398)
* first pass on profiles

* quick test

* wip

* wip

* test post thread

* record embeds

* get author feed profiles

* wip timeline

* fix get timeline

* switch from counter to tid

* tidy into a service

* quick tid test

* pr feedback

* clock -> rev

* update image formatting

* disable migration & build branch

* add recent posts to getAuthorFeed & handle post thread not found errors

* refactor for lag header

* tidy

* rm collections check

* tidy test

* pr feedback

* fix small bug

* build branch

* get migrations into system

* enable migrations
2023-08-10 13:42:31 -05:00
Paul Frazee
ab50816461
Self-labeling (#1444)
* Add self-label schemas

* switch around array and union for self-labels

* apply self-labels to post and profile views

* test for self labels in pds

* apply self-labels to post and profile views in bsky appview

* test for self labels in bsky appview

* update pds proxied test snapshots

* Add support for self-labels to the mdoeration sdk

* Disable unknown-labeler filtering until 3P support is added

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-08-09 19:06:29 -04:00
Eric Bailey
ef23263297
Merge pull request #1439 from bluesky-social/eric/app-787-media-tab
add filter param to getAuthorFeed
2023-08-09 15:38:09 -05:00
Eric Bailey
20230b46bc format 2023-08-09 14:05:03 -05:00
Eric Bailey
f41e76a653 tighten up tests 2023-08-09 14:04:23 -05:00
devin ivy
203e72f1c1
Appview preset image URLs (#1248)
* update image signing logic to presets on bsky

* update bsky config and views for preset img urls

* update bsky image tests for preset urls

* update bsky snapshots for preset image urls

* update pds proxy tests for bsky preset image urls

* tweak image paths
2023-08-09 10:21:43 -04:00
bnewbold
3ed6dceafb
remove unused/unspecified label (#1426)
* labeler: remove redundant 'underwear' label

This commit contains no change to mod policy, app behavior, etc.

We have been creating redundant "underwear" labels along with "sexual"
for many months. "sexual" is the actual label that gets actioned in mod
preferences, etc. The un-specified "underwear" labels are confusing and
take up resources on-disk and in API responses.

* reflect in appview

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2023-08-08 17:56:38 -05:00
Daniel Holmgren
4879ccb365
Appview label cache (#1451)
* add label cache to appview

* add to dev env

* fix test
2023-08-08 17:56:21 -05:00
Eric Bailey
55f33234e2 fix naming 2023-08-08 16:40:55 -05:00
Eric Bailey
b70238476f fix select cause 2023-08-08 16:38:39 -05:00
Eric Bailey
5f2fc6c1ae format 2023-08-08 16:14:27 -05:00
Eric Bailey
08ea9790d9 use where exists subquery 2023-08-08 16:13:47 -05:00
Eric Bailey
399dfdbcc5 fix tests 2023-08-08 16:03:44 -05:00
Eric Bailey
92f5979d44 update naming based on feedback 2023-08-08 15:41:48 -05:00
Eric Bailey
975b0f40ed fix lint 2023-08-08 12:22:22 -05:00
Eric Bailey
5083705a37 use more specific naming 2023-08-08 12:20:08 -05:00
Eric Bailey
59c435648d codegen again 2023-08-08 12:02:34 -05:00
Eric Bailey
ff11f5fae5 add queries and test to app view 2023-08-08 12:02:34 -05:00
Eric Bailey
7f84fb215e codegen 2023-08-08 12:02:34 -05:00
Daniel Holmgren
b16c41c2fa
Tighten feed filtering (#1443)
tighten feed filtering
2023-08-08 10:41:19 -05:00
Daniel Holmgren
bb4b4344bd
Get post hotfix (#1445)
ensure post is defined
2023-08-07 18:54:33 -05:00
Daniel Holmgren
556e438ccc
Filter out blocked posts on getPosts (#1442)
filter out blocked posts on getPosts
2023-08-07 18:52:10 -05:00
devin ivy
b5569121f8
Misc perf improvements on actor search, post deletion, actor invite codes (#1436)
* tighten fuzzy actor search for larger dataset

* add indexes to support post deletion on feed_item, listing user invites on invite_code

* fix migration whitespace

* temp comment migrations and build

* Revert "temp comment migrations and build"

This reverts commit 225d2c00cce1885f2d2cffc4724843eda4b01cee.
2023-08-07 19:38:04 -04:00
devin ivy
3f3ae4cfd5
Support proxying moderation through to appview (#1233)
* in-progress work on proxying moderation to appview

* tidy

* proxy reports pds to appview, misc tidy

* test proxying of moderation endpoints

* remove report action fkeys from pds for appview sync

* test appview/pds moderation synchronization

* tidy

* tidy

* fix admin proxy tests, build

* temp disable migration

* rm relative @atproto/api imports

* fix a couple more relative imports

* tidy

* reenable migration, comment contents temporarily

* fully enable migration, remove build, misc test fixes

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2023-08-04 16:06:04 -04:00
Daniel Holmgren
6d04bf76ef
Hive retry logic & logging (#1434)
* better logging & retry logic for hive

* tidy

* fix test

* bump timeout

* tidy pds labeler

* fix test

* build branch

* fix types

* fix types

* rm branch builds

* log hive response

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-08-03 17:42:05 -05:00
devin ivy
e5e24d510e
Push labels to PDS (#1423)
* add lexicon for unspecced applyLabels procedure

* implement label push to pds via unspecced.applyLabels

* add hive retry to bsky appview

* build

* update applyLabels to work with raw label data

* update bsky hive labeler

* remove build
2023-08-03 12:25:56 -04:00
Daniel Holmgren
450dff7fa3
Fix blocks in follow views (#1427)
* fix blocks in follow views

* check blocks between follow subjects & creators

* test
2023-08-03 10:43:30 -05:00
Foysal Ahamed
13b16b33f2
Allow setting a note when enabling/disabling invite codes (#1363)
* Allow setting a note when enabling/disabling invite codes

* Add inviteNote to repoView

*  Add test for inviteNote

*  Set to null when note is empty

*  Change note from text to varchar

* 📝 Better description
2023-08-01 17:43:21 -05:00
devin ivy
30b41029c1
Partitioning in bsky indexer (#1368)
* setup redis infra for appview indexer

* barebones bsky ingester

* add ioredis to bsky

* remove some indexer functionality from bsky api

* setup for bsky indexer

* tidy

* tidy, observe basic pipeline functioning

* process messages on bsky indexer pipeline, tidy tests and lifecycle

* trim partitions when moving cursor

* simplify config for partitions

* misc fixes for redis setup in bsky tests, add namespacing

* fix pds proxy tests

* remove cursor state from indexer partitions, simplify ingester state

* tidy

* utils for testing w/ multiple indexers, fix off-by-one xtrim

* test reingesting

* test indexer repartitioning

* add entrypoints for bsky ingester and indexer, fix db schema config, api entrypoint name, tidy

* setup and test bsky ingester backpressure, add config

* tidy

* add missing test file

* tidy redis calls, add redis sentinel config

* tidy/test some utils used in bsky pipeline

* tidy bsky pipeline tests, move helpers into dev-env

* fix pds crud test

* support redis host and password config

* better loggin/observability in ingester and indexer, make build

* add bsky ingester initial cursor config

* temporarily remove migrations from indexer/ingester

* allow ingester to batch

* packages/pg becomes packages/dev-infra with some cleanup (#1402)

* packages/dev-infra/

* Extract packages/dev-infra/_common.sh and use it

* Use period instead of source because of /bin/sh

* add docs for redis test script

* fix repartition test

* add logs to debug ci

* simplify repartitioning test, remove ci logs

---------

Co-authored-by: Jerry Chen <jerry@redelm.net>
2023-08-01 14:59:00 -04:00
Daniel Holmgren
1ad6274202
Actor service fix (#1414)
* fix up bug in actor service

* add in hydrate methods

* build fix

* couple more fixes

* tidy

* port to appview

* dont build branch

* fix issue with items in list

* tidy

* dont build branch

* quick fix
2023-08-01 12:38:14 -05:00
Daniel Holmgren
1b0a5716ce
Reflect takedowns in actor service (#1412)
reflect takedowns in actor service
2023-07-31 18:43:04 -05:00
Daniel Holmgren
7ef73467eb
Remove local whats hot from pds (#1393)
* put behind proxy header

* toggle for everyone

* tidy

* tidy

* fix
2023-07-27 19:05:12 -05:00
Daniel Holmgren
f293366e6a
Tweak timeline construction (#1367)
* add idx to post table & tweak feed construction

* comment out migration & build branch

* split out queries

* couple tweaks

* bugfix in with-friends

* lower feed data threshold

* return to subquery

* log injection on tracing

* Tweak timeline migraiton (#1369)

* v0.4.3

* tweak timeline migraiton

* misspelling

* move mutes & blocks to pds

* moar tweaks

* joins with 1 day cutoff

* no blocks/mutes on with friends

* attempt forcing custom plan

* increase feed date threshold

* trying something out

* trying something more

* tidy

* lower timeline date threshold

* tidy
2023-07-27 19:03:06 -05:00
Daniel Holmgren
030be22f60
Adjust bsky team feed (#1394)
* adjust bsky team feed

* tweak
2023-07-27 18:49:09 -05:00
Daniel Holmgren
ca7b891ece
Log injection (#1384)
datadog inject logs
2023-07-25 22:43:15 -05:00