731 Commits

Author SHA1 Message Date
Daniel Holmgren
558981ead1
Disable pds appview routes (#1644)
* wip

* remove all canProxyReadc

* finish cleanup

* clean up tests

* fix up tests

* fix api tests

* fix build

* fix compression test

* update image tests

* fix dev envs

* build branch

* fix service file

* re-enable getPopular

* format

* rm unused sharp code

* dont build branch
2023-09-22 13:51:15 -05:00
github-actions[bot]
86bbf0860a
Version packages (#1654)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-09-21 18:19:28 -07:00
Paul Frazee
56e2cf8999
Improve the types of the thread and feed preferences APIs (#1653)
* Improve the types of the thread and feed preferences APIs

* Remove unused import

* Add changeset
2023-09-21 18:10:59 -07:00
bnewbold
584dea52c4
tidy up package.json and READMEs (#1649)
* identity: README example and tidy

* tidy up package metadata (package.json files)

* updated README headers/stubs for several packages

* crypto: longer README, with usage

* syntax: tweak README

* Apply suggestions from code review

Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: devin ivy <devinivy@gmail.com>

---------

Co-authored-by: Eric Bailey <git@esb.lol>
Co-authored-by: devin ivy <devinivy@gmail.com>
2023-09-21 18:07:33 -07:00
bnewbold
1487c9f1fe
indicate that getPopular is deprecated (#1647)
* indicate that getPopular is deprecated

* codegen for deprecating getPopular
2023-09-21 14:50:23 -07:00
bnewbold
7e36c48a77
small syntax lints (#1646)
* lint: remove unused imports and variables

* lint: prefix unused args with '_'

* eslint: skip no-explicit-any; ignore unused _var (prefix)

* eslint: explicitly mark ignores for tricky cases
2023-09-21 10:07:54 -07:00
bnewbold
462de219ee
api: update login/resumeSession examples in README (#1634)
* api: update login/resumeSession examples in README

* Update packages/api/README.md

Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>

---------

Co-authored-by: Daniel Holmgren <dtholmgren@gmail.com>
2023-09-20 19:15:12 -07:00
github-actions[bot]
49278430e0
Version packages (#1640)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-09-20 11:12:04 -07:00
Paul Frazee
061865c8bf
Add feed-vew and thread-view preferences (#1638)
* Add feed and thread preference lexicons

* Add feed-view and thread-view preference APIs
2023-09-20 10:22:27 -07:00
Daniel Holmgren
828dfa8a8a
Enforce properties field on lexicon object schemas (#1628)
* add empty properites to thread gate schema fragments

* tweak lexicon type
2023-09-19 10:51:26 -05:00
github-actions[bot]
38f8f44c41
Version packages (#1612)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-09-15 15:49:31 -05:00
github-actions[bot]
d7d7a043de
Version packages (#1602)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2023-09-14 17:53:14 -05:00
Daniel Holmgren
9879ca97b7
Interaction Gating (#1561)
* lexicons for block lists

* reorg blockset functionality into graph service, impl block/mute filtering

* apply filterBlocksAndMutes() throughout appview except feeds

* update local feeds to pass through cleanFeedSkeleton(), offload block/mute application

* impl for grabbing block/mute details by did pair

* refactor getActorInfos away, use actor service

* experiment with moving getFeedGenerators over to a pipeline

* move getPostThread over to a pipeline

* move feeds over to pipelines

* move suggestions and likes over to pipelines

* move reposted-by, follows, followers over to pipelines, tidy author feed and post thread

* remove old block/mute checks

* unify post presentation logic

* move profiles endpoints over to pipelines

* tidy

* tidy

* misc fixes

* unify some profile hydration/presentation in appview

* profile detail, split hydration and presentation, misc fixes

* unify feed hydration w/ profile hydration

* unify hydration step for embeds, tidy application of labels

* setup indexing of list-blocks in bsky appview

* apply list-blocks, impl getListBlocks, tidy getList, tests

* tidy

* update pds proxy snaps

* update pds proxy snaps

* fix snap

* make algos return feed items, save work in getFeed

* misc changes, tidy

* tidy

* fix aturi import

* initial lexicons for interaction-gating

* add interactions view to post views

* codegen

* model bad reply/interaction check state on posts

* initial impl for checking bad reply or interaction on write

* omit invalid interactions from post thread

* support not-found list in interaction view

* hydrate can-reply state on threads

* present interaction views on posts

* misc fixes, update snaps

* tidy/reorg

* tidy

* split interaction gating into separate record in lexicon

* switch interaction-gating impl to use separate record type

* allow checking reply gate w/ root post deletion

* fix

* initial gating tests

* tighten gated reply views, tests

* reply-gating list rule tests

* allow custom post rkeys within window

* hoist actors out of composeThread()

* tidy

* update thread gate lexicons, codegen

* lex fix

* rename gate to threadgate in bsky, update views

* lex fix

* improve terminology around reply validation

* fix down migration

* remove thread gates on actor unindexing

* add back .prettierignore

* tidy

* run ci on all prs

* syntax

* run ci on all prs

* format

* fix snap

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-09-14 15:24:48 -05:00
Daniel Holmgren
578757cb44
List feeds (#1557)
* lexicons for block lists

* reorg blockset functionality into graph service, impl block/mute filtering

* apply filterBlocksAndMutes() throughout appview except feeds

* update local feeds to pass through cleanFeedSkeleton(), offload block/mute application

* impl for grabbing block/mute details by did pair

* refactor getActorInfos away, use actor service

* experiment with moving getFeedGenerators over to a pipeline

* move getPostThread over to a pipeline

* move feeds over to pipelines

* move suggestions and likes over to pipelines

* move reposted-by, follows, followers over to pipelines, tidy author feed and post thread

* remove old block/mute checks

* unify post presentation logic

* move profiles endpoints over to pipelines

* tidy

* tidy

* misc fixes

* unify some profile hydration/presentation in appview

* profile detail, split hydration and presentation, misc fixes

* unify feed hydration w/ profile hydration

* unify hydration step for embeds, tidy application of labels

* setup indexing of list-blocks in bsky appview

* apply list-blocks, impl getListBlocks, tidy getList, tests

* tidy

* update pds proxy snaps

* update pds proxy snaps

* fix snap

* make algos return feed items, save work in getFeed

* misc changes, tidy

* tidy

* fix aturi import

* lex

* list purpose

* lex gen

* add route

* add proxy route

* seed client helpers

* tests

* mutes and blocks

* proxy test

* snapshot

* hoist actors out of composeThread()

* tidy

* tidy

* run ci on all prs

* format

* format

* fix snap name

* fix snapsh

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-09-14 11:19:04 -05:00
Eric Bailey
3877210e7f
add getSuggestedFollowsByActor (#1553)
* add getSuggestedFollowsByActor lex

* remove pagination

* codegen

* add pds route

* add app view route

* first pass at likes-based suggested actors, plus tests

* format

* backfill with suggested_follow table

* combine actors queries

* fall back to popular follows, handle backfill differently

* revert seed change, update test

* lower likes threshold

* cleanup

* remove todo

* format

* optimize queries

* cover mute lists

* clean up into pipeline steps

* add changeset
2023-09-13 15:56:12 -05:00
Daniel Holmgren
07bb0da0cf
Remove temp.upgradeRepoVersion (#1588)
remove temp.upgradeRepoVersion
2023-09-12 12:37:57 -05:00
Daniel Holmgren
3cdc72a0a7
Fix getRepo since (#1579)
* fix schemas

* test

* format
2023-09-12 12:37:45 -05:00
devin ivy
6f419790ac
Feature: block lists (#1531)
* lexicons for block lists

* reorg blockset functionality into graph service, impl block/mute filtering

* apply filterBlocksAndMutes() throughout appview except feeds

* update local feeds to pass through cleanFeedSkeleton(), offload block/mute application

* impl for grabbing block/mute details by did pair

* refactor getActorInfos away, use actor service

* experiment with moving getFeedGenerators over to a pipeline

* move getPostThread over to a pipeline

* move feeds over to pipelines

* move suggestions and likes over to pipelines

* move reposted-by, follows, followers over to pipelines, tidy author feed and post thread

* remove old block/mute checks

* unify post presentation logic

* move profiles endpoints over to pipelines

* tidy

* tidy

* misc fixes

* unify some profile hydration/presentation in appview

* profile detail, split hydration and presentation, misc fixes

* unify feed hydration w/ profile hydration

* unify hydration step for embeds, tidy application of labels

* setup indexing of list-blocks in bsky appview

* apply list-blocks, impl getListBlocks, tidy getList, tests

* tidy

* update pds proxy snaps

* update pds proxy snaps

* fix snap

* make algos return feed items, save work in getFeed

* misc changes, tidy

* tidy

* fix aturi import

* hoist actors out of composeThread()

* tidy

* run ci on all prs

* format

* build

* proxy graph.getListBlocks

* remove unneeded index

* build pds

* setup noop listblock indexer on pds

* remove build

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2023-09-12 12:07:31 -04:00
Paul Frazee
841e906101 @atproto/api@0.6.12 2023-09-11 13:24:08 -07:00
Paul Frazee
4536967280
Fixes and updates to the preferences API (#1575)
* Fix to handle duplicate preference key entries

* Add personal details preference API to sdk

* Add Array.prototype.findLast() type declaration

* Move interface declaration to ensure it's included in other package builds
2023-09-11 13:23:24 -07:00
Paul Frazee
d1df1f3232 @atproto/api@0.6.11 2023-09-08 10:37:24 -07:00
Paul Frazee
44b721b6ae
Add personal-details user preference with birth date (#1565)
* Add personal-details user preference with birth date

* Add personal details pref to settings union
2023-09-08 10:35:49 -07:00
Eric Bailey
a7c42cfe39
Clean up prettier/eslint scripts (#1514)
* add prettierignore, hoist script

* upgrade prettier

* bump prettier eslint deps

* format all files

* hoist prettier check

* remove unused scripts, hoist lint:fix

* remove npm-run-all, unused

* hoist lint

* remove lint scripts

* improve lint scripts

* remove prettierignores

* downgrade prettier deps to fix codgen

* reformat all files

* update makefile, format

* fix locklife

* final format

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2023-09-06 19:27:50 -05:00
David Buchanan
90b0a9d809
RFC: Support image aspect ratio metadata (#1306)
* Add aspectRatio field to embed image lexicon

* lexicon codegen

* pass aspectRatio through to imagesEmbed views

* simplify aspectRatio logic, port to bsky package

* re-run lexicon codegen
2023-09-06 10:08:06 -05:00
Eric Bailey
90e8325e5f
pnpm setup (#1532)
* use pnpm

* fix dependency issues, replace yarn and lerna scripts

* remove the main/dist scripts

* update Dockerfiles

* use pnpm

* fix dependency issues, replace yarn and lerna scripts

* remove the main/dist scripts

* update Dockerfiles

* update bin script

* remove unused zod dep

* fix type errors in pds

* add types prop to packages

* remove unused, bump lock

* fix test running

* build before test

* fix pino types

* format

* pds depends on dev-env in test

* refer to src instead of built packages

* pds relies on bsky in test too

* remove yarn.lock

* add -r flag to root test

* test push to aws

* remove docker test

* add publishConfig to new package

* move services to top level dir

(cherry picked from commit f5012bec33435a4473e9960066807623334f3aff)

* update workflow paths

(cherry picked from commit 5c70f0176d381ca35d6be10cfa173e22373a5b5d)

* add main-to-dist script

* use script in all packages, remove old Dockerfiles

* remove old bsky service

* remove newline

* test container builds

* Revert "test container builds"

This reverts commit c228611f5e8e1624d4b124be4976c49590130f43.

* remove unused config

* test build containers

* pnpm in syntax

* bump dd-trace

* shamefully hoist

* even more shame

* hoist, externalize deps

* clean install for prod and smaller containers

* dont build branches

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
2023-09-05 18:45:49 -05:00
Foysal Ahamed
4fb32e1e06
Add missing labels that are available on the moderation tool (#1515)
* Add missing labels that are available on the moderation tool

* Remove unused label

* More cleanup

* 🧹 Get rid of ncii as well

*  Add new label in json instead of generated file 🤦🏽‍♂️
2023-09-05 10:49:55 -04:00
Eric Bailey
16662b220d
@atproto/api@0.6.9 (#1546)
* @atproto/api@0.6.9

* @atproto/api@0.6.10
2023-09-01 10:43:30 -05:00
Eric Bailey
f84027fe8e
add getFeedSuggestions (#1542)
* add getFeedSuggestions lex

* move to feed namespace

* codegen

* add table and migration

* add endpoints

* remove comment

* add test

* fix order

* rename

* just remove proxy check
2023-08-31 17:19:34 -05:00
Paul Frazee
6d765f7f61 @atproto/api@0.6.8 2023-08-29 17:13:35 -07:00
Paul Frazee
8db9077c68 @atproto/api@0.6.7 2023-08-29 17:10:13 -07:00
Paul Frazee
2b8cbcee1f
Add preferences helpers to SDK (#1537) 2023-08-29 17:09:46 -07:00
Daniel Holmgren
ad1fcf1387
Repo history rewrite (#1479)
* logical changes in repo

* tests

* tweak commit data

* building pds

* patching up some tests

* tidy + more tests

* patch up bsky

* clean up db

* db migration

* small patches

* fix up another test

* reinclude prevs

* api & lex updates

* add back in deprecated routes

* backward compatibility for commit v2

* add upgrade repo version root

* move namespace

* migration test

* patch up a few more tests

* remove deprecated rebase routes

* tweak api

* sprinkle rev around a few more places

* getCurrent -> getLatestCommit

* fix test

* pr feedback & tidy

* fix up block pagination

* tidy again

* add in a tets

* add cursor to listBlobs

* getRepo rev -> since

* clean up proofs test

* dont change getHead

* tweak migrate route

* build branch

* hit index in block range query

* check for dupe record refs

* tidy

* set prev to null

* dont build branch
2023-08-29 19:07:21 -05:00
bnewbold
bd3966555f
@atproto/syntax: merge of @atproto/identifier, uri, and nsid (#1464)
* move nsid package in to identifier

* move uri package in to identifier

* update packages to pull from identifier, not nsid or uri

* rename @atproto/identifier to @atproto/syntax

* update all refs from @atproto/identifer to @atproto/syntax
2023-08-25 10:18:10 -05:00
Paul Frazee
1dd0c46593 @atproto/api@0.6.6 2023-08-23 16:18:04 -07:00
Ansh
eae682f3bf
Native Notifications (#1428)
* pseudocode for sending notification to user

* add notification push token table

* lexicon codegen

* `pds` and `api` codegen

* update lexicon

* add simple function to `putNotificationPushToken` into `notification_push_token` table

* reorgnaize imports

* add unspecced `putNotificationPushToken` to pds

* add `notification-push-tokens` table to PDS

* check if token exists before adding it to db

* add endpoint and appId to PDS table

* setup notification server

* fix logic for inserting token into db

* fix NotificationServer methods by making them static

* fix merge conflicts

* add comments on how sending ntoifications work

* remove dead code

* move notifServer to AppContext

* refactor code to handle notif sending in `Indexer` service

* add additional data when sending notifs

* clean up code

* move notif logic to indexer

* add `appId` and `endpoint` optional params to putNotificationPushToken

* clone notification code to AppView

* add endpoint to register push token with app view

* disable pds and enable app view notification server

* clean up code

* simply logic to check if a token already exists

* remove NotificationServer from PDS

* remove notification-push-token table from PDS

* remove `putNotificationPushToken` endpoint

* clean up code

* let `axios` throw error if `gorush` has an error

* let `kysely` throw error if notif cannot be registered by client

* rename `registerPushNotificationEndpoint` to `registerPushNotifications`

* delete `putNotificationPushToken` from AppView

* rename putPushNotificationToken to registerPushNotification

* remove dead notification code from pds

* remove sanitizeDisplayName from NotificationSever

* move `pushNotificationEndpoint` to config

* temp add `pushNotificationEndpoint` to dev-env setup

* remove example test from feed-generation.test.ts

* add test for registerPushNotification and clean up error handling

* move notifications test to its own file

* add test for NotificationServer to check if tokens are retrieved correctly

* remove unused functions from NotificationServer

* add additional tests for NotificationServer

* add return type to getNotificationDisplayAttributes function

* remove unnecessary comments

* remove dead NotificationServer code from PDS

* clean up code to prepareNotifsToSend

* put sending notifications as part of the backgroundQueue

* log instead of throwing error if notification attributes don't exist

* remove logs

* add more tests to `notification-server.test.ts`

* show replied with text for reply notifs

* better error handling when sending notifications via backgroundQueue

* add rate limit and batching to sending notifications

* add comments to NotificationServer

* merge with main

* use redis for rate limiting instead of normal rate limits

* move `notificationBatchSize` into config

* usePrimary db in test

* hoist push notif migration to present, update model to remove endpoint

* update push notif lexicon

* pare down lex for unregistering push

* helpers for working with service endpoints from did doc

* service-authed register/unregister push notifs

* add well-known endpoint to appview

* update bsky notif service tests

* fix to batching logic, misc tidy

* batch display notifications

* colocate all notification sending logic

* tidy tests

* remove unregister token for now

* fix registerPush lexicon, make a procedure

* fix registerPush impl, test pds proxy to notif service

* fix tests, make notif server optional when not configured

* fix notif server config for bsky app service

* move notif server rate limiting in-mem for now, add sending retry

* codegen tidy

* only push notifs on commit

* build

* fix notif rate limiter check

* send notifs from users w/o a profile

* remove build

---------

Co-authored-by: dholms <dtholmgren@gmail.com>
Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-08-23 18:37:14 -04:00
Paul Frazee
7e4edb2ce1 @atproto/api@0.6.5 2023-08-22 12:13:26 -07:00
Eric Bailey
98cc8c5214 codegen 2023-08-22 11:16:56 -05:00
Eric Bailey
048cd16d0c Merge remote-tracking branch 'origin' into eric/app-796-likes-tab
* origin: (21 commits)
  styleguide
  Increase CI test matrix size (#1490)
  tweak author-feed rejects test syntax
  update test
  format
  add repost of reply, update snaps
  simplify query
  fix typos
  include reposted replies in posts_no_replies filter
  fix bad test syntax
  fix actor takedown tests
  use getActor, failing atm
  only include media posts by post creator
  Fix condition for viewing soft-deleted followers (#1485)
   Expose takendown profile, their follows and followers to mods (#1456)
  Handle revalidation (#1474)
  Handle db pool errors on appview (#1483)
  Handle db client errors on appview (#1481)
  v0.6.4
   Allow creating moderation action with a duration (#1431)
  ...
2023-08-18 11:10:04 -05:00
Devin Ivy
ea9d96e3a4 v0.6.4 2023-08-15 17:12:30 -04:00
Foysal Ahamed
7bc701e2e1
Allow creating moderation action with a duration (#1431)
*  Add action duration to action model

*  Add periodic moderation action reversal

*  Use actionDurationInHours and remove takedownExpiresAt

*  Destroy mod action reversal job

*  Add test for automatic moderation action reversal

* 🧹 Port over actionDuration to bsky package

* ♻️ Better naming for action duration and minor fixes

* ♻️ Optionally build expiresAt based on createdAt when logging action

* ♻️ Adjust constructor params

*  Copy over migration from pds to bsky package

* 🚨 Fix indentation

*  Propagate action reversal from app-view to pds

*  Revert locally only if pushAgent is not found

* build

* merge dbcoordinator work into periodic mod actions

* fix entrypoint for periodic mod reversals

* do not run periodic reversals on sequencer leader

* fix duration in action detail on bsky

* update env for pushing mod reversals

* allow zero-duration actions for testing

* remove build

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-08-15 15:44:53 -04:00
Eric Bailey
23d6feb3c9 add method to bsky agent 2023-08-15 13:58:40 -05:00
Eric Bailey
b05986f7d1 codegen 2023-08-14 15:39:48 -05:00
Paul Frazee
9a824aeae6 @atproto/api@0.6.3 2023-08-10 15:37:50 -07:00
Daniel Holmgren
244bf46e74
Include limited info on blocked embeds (#1463)
* add block info to embeds

* fix codegen

* Correctly handle blocked embeds and add block-other cause

* update snaps

* Correctly identify blocking behavior in embeds

---------

Co-authored-by: Paul Frazee <pfrazee@gmail.com>
2023-08-10 15:54:24 -05:00
Paul Frazee
29a1ee791a @atproto/api@0.6.2 2023-08-10 12:53:35 -07:00
Daniel Holmgren
ba979b4cdb
Include limited viewer info on blocked post (#1457)
* include limited viewer info on block

* blockedAuthor schema

* make author required

* merge & update codegen

* patch up tests
2023-08-10 14:31:45 -05:00
Daniel Holmgren
3f0e1b2563
Read after write (#1398)
* first pass on profiles

* quick test

* wip

* wip

* test post thread

* record embeds

* get author feed profiles

* wip timeline

* fix get timeline

* switch from counter to tid

* tidy into a service

* quick tid test

* pr feedback

* clock -> rev

* update image formatting

* disable migration & build branch

* add recent posts to getAuthorFeed & handle post thread not found errors

* refactor for lag header

* tidy

* rm collections check

* tidy test

* pr feedback

* fix small bug

* build branch

* get migrations into system

* enable migrations
2023-08-10 13:42:31 -05:00
Paul Frazee
b406a45829 @atproto/api@0.6.0 2023-08-09 16:08:05 -07:00
Paul Frazee
ab50816461
Self-labeling (#1444)
* Add self-label schemas

* switch around array and union for self-labels

* apply self-labels to post and profile views

* test for self labels in pds

* apply self-labels to post and profile views in bsky appview

* test for self labels in bsky appview

* update pds proxied test snapshots

* Add support for self-labels to the mdoeration sdk

* Disable unknown-labeler filtering until 3P support is added

---------

Co-authored-by: Devin Ivy <devinivy@gmail.com>
2023-08-09 19:06:29 -04:00
Eric Bailey
ef23263297
Merge pull request #1439 from bluesky-social/eric/app-787-media-tab
add filter param to getAuthorFeed
2023-08-09 15:38:09 -05:00