1093 Commits

Author SHA1 Message Date
Foysal Ahamed
8857fb0845
Filter reports by reporter did (#1165)
*  Hydrate subject author's repo in moderation reports

*  Allow filtering moderation reports by reporter DIDs

* 🚧 Expose only handle of the subject repo on moderation reports

* 🚧 Regenerate lexicons off main

* 🧹 Cleanup unnecessary joins

*  Updating snapshots

*  Make handle optional to allow serving reports even if handle doesn't exist

*  Port over subjectHandle implementation to bsky package

*  Port over filter by reporter implementation to bsky package

* Simplify query builder

Co-authored-by: devin ivy <devinivy@gmail.com>

* Simplify query builder

Co-authored-by: devin ivy <devinivy@gmail.com>

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-06-09 09:21:09 -04:00
Foysal Ahamed
9a2dc7119e
Hydrate subject author's repo in moderation reports (#1135)
*  Hydrate subject author's repo in moderation reports

* 🚧 Expose only handle of the subject repo on moderation reports

* 🚧 Regenerate lexicons off main

* 🧹 Cleanup unnecessary joins

*  Updating snapshots

*  Make handle optional to allow serving reports even if handle doesn't exist

*  Port over subjectHandle implementation to bsky package
2023-06-08 10:17:52 -04:00
Daniel Holmgren
35e035dad2
Disable resolution of handles through xrpc (#1179)
* disable resolution of handles through xrpc

* tweak admin.updateHandle
2023-06-07 16:00:52 -05:00
devin ivy
dd38c8b3b5
Normalize creation times on list, list item, feedgen records (#1178)
Normalize creation times on lists, list items, feedgens
2023-06-07 16:58:15 -04:00
devin ivy
ff78a30b9e
Ensure keyset pagination makes use of indexes on timeline (#1174)
Alternative pagination query snippet for better use of indexes
2023-06-07 11:46:33 -04:00
devin ivy
a23fb1eac3
Fix lists pagination, ensure paginating on correct column (#1175)
Ensure lists pagination uses createdAt both in query and for building cursor
2023-06-07 11:41:02 -04:00
Daniel Holmgren
6fb9a8faea
Fix dropped events in sync.subscribeRepos (#1167)
* wip

* first pass done

* chnage around how channels function

* bugfixing

* fix up migration & tests

* fix dev-env

* add prep migration for stream fixing

* application code changes to use new repo_event table

* dont send evts

* add outgoing_repo_seq table + sqlite migrations

* patch up down migration

* patching up a couple things

* update test

* patchup

* re-enable subscription route

* add outgoingSeq column

* updating

* accidental add

* temporarily disable stream

* resolve merge conflict

* add migration

* migration for seq

* fix test

* id -> seq

* get schema lock ids

* tidy

* fix server test

* fix up

* use bsky uniqueLockId approach

* tidy

* gh actions build

* build branch

* build branch

* tweak test

* rm constraint rename

* small bugfix

* check for seq not null

* fix up issues

* one more small change

* tidy

* no build branches
2023-06-06 19:18:44 -05:00
Foysal Ahamed
46e1f051e1
Allow searching repo by DID (#1084)
*  Allow searching repo by DID

*  Port over did search from pds

*  Add tests for bsky implementation of admin searchRepos

* 🧹 Cleanup according to PR review
2023-06-06 00:12:14 -04:00
bnewbold
a4a4af1693
all packages: prettier applies to tests/ in addition to src/ (#1170)
* pds: prettier:fix (on a JSON file)

* package.json (multiple): run prettier on tests/

* bsky: prettier:fix (on a JSON file)
2023-06-05 19:26:32 -07:00
devin ivy
c20ed66f2a
Optimization for label subqueries on feeds (#1164)
* temp disable hot-classic

* Switch labels in whats-hot to values list

* Switch labels in getPopular (legacy hot-classic) to values list

* Reenable hot-classic, switch labels to values list

* remove branch build in workflow
2023-06-03 14:10:57 -04:00
Daniel Holmgren
bf4ac79cef
Fix up what's hot (#1133)
* do whats hot off of candidate feed

* tidy
2023-06-01 21:26:40 -05:00
devin ivy
53a107b237
Fix pagination limit when not respected by feedgens (#1129)
* Enforce limit param in the case that the feedgen does not

* Tidy
2023-06-01 17:36:22 -04:00
Daniel Holmgren
243972bd69
Rebase indices (#1126)
* prepping for rebases

* add down migration
2023-06-01 15:34:41 -05:00
devin ivy
feb320c93e
Revert perf tweak to with-friends (#1125)
Revert change to with-friends
2023-05-31 12:37:40 -04:00
devin ivy
4e592ef7f8
Perf tweaks to with-friends, mutuals, label batching (#1124)
* Tweak with-friends and mutuals queries to be less constrained for planner, omit reposts in mutuals feed

* More batching of label queries
2023-05-31 11:09:45 -04:00
Daniel Holmgren
756658454d
Mount admin.rebaseRepo (#1121)
hook up admin rebase route
2023-05-30 15:53:58 -05:00
devin ivy
95252ddb4f
Batch fetching of labels across posts and actors (#1117)
* Rename getLabelsForSubjects to getLabelsForUris

* Generic method for getting profiles organized by did or aturi subject

* Batch label fetching for posts and authors in feeds and threads

* Tidy
2023-05-30 13:23:33 -05:00
Daniel Holmgren
87fc4ce1bc
Return custom suggestions (#1118)
* return custom suggestions on server

* allow ordered suggestions
2023-05-30 13:23:22 -05:00
devin ivy
3c9646bee6
Revert feed date threshold for with-friends (#1120)
Revert feed date threshold for with-friends, not beneficial
2023-05-30 14:14:43 -04:00
Foysal Ahamed
4a6c9764e3
Allow reverting chronological order of moderation reports (#1113)
*  Allow reverting chronological order of moderation reports

*  Port over report sorting to bsky package
2023-05-30 11:50:46 -04:00
devin ivy
01cb3831f4
Followup to getRepo and getTimeline perf tweaks (#1116)
* To avoid performing many view refreshes in a row w/ different leaders, schedule them upfront

* Apply feed threshold to with-friends and mutuals algos

* Revert increase to getRepo commit chunk size

* Alter feed threshold approach to be tighter and relative to cursor
2023-05-30 11:46:11 -04:00
devin ivy
a900e955a6
Disallow scheduling repeated view refreshes (#1114)
To avoid performing many view refreshes in a row w/ different leaders, schedule them upfront
2023-05-29 14:40:45 -04:00
Daniel Holmgren
39cde47c11
Admin rebases (#1112)
add admin route for kicking off rebases
2023-05-29 12:42:39 -05:00
devin ivy
6ee810c6d8
Minor perf tweaks for getTimeline and getRepo (#1111)
* Avoid long scans for users w/ sparse feeds on getTimeline

* Increase getRepo commit chunk size
2023-05-29 11:33:46 -04:00
Foysal Ahamed
3414fcedbf
Allow ignoring moderation reports by subject (#1091)
*  Allow ignoring moderation reports by subject

* 🔧 Remove unwanted package.json change

*  Remove subject format specifier

*  Port over ignore subject implementation to bsky package

* Revert " Port over ignore subject implementation to bsky package"

This reverts commit 3c782c1805548d1ebdbbd08b68b6e92ac97fdee5.

* 🚧 Port over ignore subject implementation to bsky package

*  Generate lexicons for bsky
2023-05-29 00:39:03 -04:00
devin ivy
237bb97e20
Log backgrounded error when refreshing did cache (#1110)
* Log backgrounded error when refreshing did cache

* remove branch build
2023-05-29 00:23:17 -04:00
Daniel Holmgren
693c703d78
Skip popular with friends test (#1109)
skip test
2023-05-26 18:48:35 -05:00
Daniel Holmgren
ec15af5de8
Algo tweaks (#1108)
* Temporarily skip online/valid checks for feedgens while external feedgens catch-up

* tweak algos

* hook up did resolver with cache

* Re-enable whats-hot w/ materialization

* pulling a fast one on the query optimizer

* clean up

* disable branch build

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-05-26 18:27:23 -05:00
devin ivy
d00e3112d0
Update pool size for view maintainer (#1107)
Update pool size for view maintainer, prettier format
2023-05-26 19:19:58 -04:00
Daniel Holmgren
43e83a8681
Improving mutuals again (#1106)
* another go at it

* one more go at it

* add self to best of follows
2023-05-26 17:34:15 -05:00
Daniel Holmgren
a9c70050fd
Improve mutuals feed (#1105)
fix
2023-05-26 16:25:07 -05:00
Daniel Holmgren
12dc71931a
Best of follows feed (#1102)
add best of follows feed
2023-05-26 15:02:36 -05:00
devin ivy
212a3a7456
Improve logging around materialized view refresh (#1104) 2023-05-26 16:02:01 -04:00
Daniel Holmgren
9489194841
Mutuals feed (#1101)
add mutuals feed
2023-05-26 15:01:18 -05:00
devin ivy
9eb817d488
Materialize candidates for whats-hot algo (#1086)
* Setup whats-hot materialized view w/ params, run view maintainer in service entrypoint

* Update whats-hot to use new materialized view, tidy

* Update migration to create with no data

* Revert whats-hot migration change for no data

* Bump refresh rate for view down to 1min
2023-05-26 15:05:53 -04:00
devin ivy
24d50e8f33
Standalone migration for whats-hot algo materialized view (#1100) 2023-05-26 14:20:39 -04:00
devin ivy
549688a67a
Implement quicker mute checks on pds (#1098) 2023-05-26 12:49:43 -05:00
Daniel Holmgren
03a74b4259
Catch missed txn rollbacks (#1094)
* catch missed rollbacks

* change approach

* revert back to simple way

* test

* add to bsky package as well

* improve handler & tests
2023-05-26 11:04:26 -05:00
devin ivy
7a7c9c75af
Improved custom algo debugging (#1096)
* Better feedgen errors in getFeed and getFeedGenerator

* Support handler output headers in codegen

* Support handler output headers in xrpc-server

* Implement server-timing helpers and apply them to getFeed

* fix codegen issue

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2023-05-26 11:59:05 -04:00
Miles Zimmerman
5ca6ce31e1
Increase flexibility of paramaters accepted by DB_CREDS_JSON env var (#974) 2023-05-25 21:02:20 -05:00
Daniel Holmgren
41330ca6b0
Rebase rework (#1095)
* rework rebases a bit

* include did in err

* move formatting outside of txn

* reorg after rebase processing
2023-05-25 19:23:47 -05:00
David Buchanan
c62964b7b4
Limit getPostThread recursion into parents (#1062)
* limit getPostThread recursion into parents

* add parentHeight parameter

* Lexicons: specify default/min/max values for depth and parentHeight, for getPostThread

* prettier formatting

* re-run lexicon codegen

* do the rest of lexicon codegen (oops)

* add 'shallow ancestors' fetch test
2023-05-25 19:22:05 -05:00
Daniel Holmgren
0307340c5a
Label after write (#1067)
* label imgs after write completes

* in on commit fn

* dont do in commit hook
2023-05-25 19:14:12 -05:00
Daniel Holmgren
743eaf1cf0
Move handle resolution to .well-known (#1048)
* move handle resolution to .well-known

* required handle on resolveHandle

* rm test

* tidy

* tidy

* fix up appview

* missing await

* atproto-handle -> atproto-did

* shift did & handle resolution to new identity package

* fix up network mocks

* fix up another test

* one more

* drop lex comment

* rm handle param

* Update packages/identity/src/handle/index.ts

Co-authored-by: devin ivy <devinivy@gmail.com>

* still temporarily support xrpc handle resolution

* typo

* ensure return value is a string

---------

Co-authored-by: devin ivy <devinivy@gmail.com>
2023-05-25 19:04:10 -05:00
dholms
85bcd18a7b v0.1.11 2023-05-24 17:43:02 -05:00
Daniel Holmgren
5df2903061
Feed publishing tweaks (#1087)
* add feed gen did to cfg

* allow puts for generator records

* add a quick test

* quick test for updates

* fix up test
2023-05-24 17:38:01 -05:00
devin ivy
53044ae48e
Get popular feedgens (#1090)
* Lexicon for unspecced popular feed gen method

* Endpoint impl for popular feed gens
2023-05-24 17:37:53 -05:00
devin ivy
f7c502b145
Improve performance of whats-hot custom algo (#1078)
Improve whats-hot algo perf by reorganizing query, excluding some unpopular results more proactively
2023-05-23 23:01:51 -04:00
Ilya Siamionau
a22d56aa21
Fix lexicon. "format" instead of "ref" in type "string" (#1080)
* Fix lexicon

* codegen types
2023-05-23 16:06:03 -04:00
devin ivy
7e22719967
Tweaks to with-friends algo (#1082)
* Iteration on with-friends algo

* Fix with-friends tests
2023-05-23 15:50:45 -04:00