feat/sv-searching-txtai #2

Closed

dufok wants to merge 44 commits from feat/sv-searching-txtai into dev

pull from: feat/sv-searching-txtai

merge into: discours.io:dev

discours.io:dev

discours.io:feature/auth-internal

discours.io:feat/sv-search-v4

discours.io:fix/sv-authorSortingBy

discours.io:staging

discours.io:main

discours.io:feature/branched-pagination

discours.io:feature/caching-improved

Conversation 0 Commits 44 Files Changed 11 +1060 -228

dufok commented 2025-04-07 14:42:59 +00:00

Owner

Copy Link

working backend search engine on txtai with preinit on start

working backend search engine on txtai with preinit on start

dufok added 39 commits 2025-04-07 14:42:59 +00:00

feat(search.py): change to txtai server, with ai model. And fix granian workers d55448398d

debug: fixed workflows gitea e6a416383d

debug: fixed workflows gitea 59e71c8144

feat(db.py): added fetch_all_shouts, to populate the search index c0b2116da2

feat: moved txtai and search procedure in different instance f249752db5

feat: with logs >>> a9c7ac49d6

debug: something wrong one stap back with logs 24cca7f2cb

debug: add logs in search.py and change and input validation ... index ver too 39242d5e6c

debug: no redis for indexing in nackend side ad0ca75aa9

debug: start index 90699768ff

debug: with logs in indexing procedure 385057ffcd

debug: more logs in indexing 57e1e8e6bd

debug: Limit max chars for bulk indexing 19c5028a0c

debug: change logs entris and indexing not wraps all in documents eb4b9363ab

feat(search.py): documnet for bulk indexing are categorized 50a8c24ead

debug: decrease chars in search.py for bulk indexing ebb67eb311

feat(search.py): with db reset endpoint f1d9f4e036

debug(search.py): encrease batch size for bulk indexing fb820f67fd

debug(search.py): encrease batch size for bulk indexing 316375bf18

refactor(search.py): moved initialization logic in search-txtai instance 60a13a9097

debug: server.py -> threds 1 , search.py -> add 3 times reconect 077cb46482

feat: give little timeout for resource stab 6e284640c0

debug(server.py): with 4 workers (threds). cheking reindexing 88012f1b8c

feat: with index sync endpoints configs 1fd623a660

debug(search.py): with more logs when check sync of indexing f089a32394

feat(search.py): detects both missing documents and null embeddings 7f36f93d92

refactor(search.py): moved to use one table docs for embdings and docs store e405fb527b

debug(reader.py: trying back shout id in query call 34a354e9e3

feat(type.qraphql): SearchResult with shout id ae85b32f69

debug(reader.py): trying to handle main topic ids founded ff3a4debce

debug(reader.py): query_with_stat(info) always 0bc55977ac

refactor(reader.py): rm debug line 9ebb81cbd3

merged with dev 9a02ca74ad

refactor(reader.py): Remove the unnecessary topic joins that cause duplicate results ecc443c3ad

feat: add cash for storing searchresalts and hold them for working pagination. Now we are have offset for use on frontend a0db5707c4

fix(search.py): store all results in cash not only first offset bc4ec79240

fix(reader.py): change sorting and answer on querys 78326047bf

fix(reader): sorting by rang not by id in cash c533241d1e

debug: without debug logging. clean 106222b0e0

to requested review from to 2025-04-13 05:35:59 +00:00

to added 1 commit 2025-04-13 05:36:20 +00:00

Merge branch 'dev' into feat/sv-searching-txtai 83d61ca76d

dufok added 1 commit 2025-04-15 22:21:35 +00:00

Merge branch 'dev' into feat/sv-searching-txtai ...

:

e382cc1ea5

dufok added 1 commit 2025-04-15 22:32:52 +00:00

feat: title weight procedure 27b0928e73

dufok force-pushed feat/sv-searching-txtai from 27b0928e73 to e382cc1ea5 2025-04-16 20:35:32 +00:00 Compare

dufok added 3 commits 2025-04-23 21:25:19 +00:00

feat(search.py): separate indexing of Shout Title, shout Body and Authors 4d965fb27b

merge dev ec9465ad40

feat: with three separate endpoints 11654dba68

dufok closed this pull request 2025-05-20 13:55:32 +00:00

All checks were successful

Hide all checks

Deploy on push / deploy (push) Successful in 5s

Details

Pull request closed

This pull request cannot be reopened because the branch was deleted.

Sign in to join this conversation.

Reviewers

No reviewers

to

Labels

Clear labels

No items

No Label

Milestone

Clear milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

No Assignees

2 Participants

Notifications

Subscribe

Due Date

The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: discours.io/core#2

No description provided.