Skip to content

gaby: exclude stale(?) web document #63

Closed

Description

@hyangah

From golang/go#67901 (comment)

Docs like https://go.dev/doc/go1.17_spec#Package_initialization are kept for historical purposes.
We may come up with a workaround for this specific issue. I am not sure about general solutions.

Some approaches I am thinking of:

  • Label such docs manually in the document source and exclude them

  • Label such docs using LLM (e.g. "obsolete"?) and exclude them

    (we can also do the same for issues that we don't want to appear in the related info by labelling/classifying appropriately)

  • Before posting, drop almost duplicates (e.g. by checking pair-wise similarity comparison)

Activity

  1. gopherbot commented on Dec 3, 2024

    @gopherbot
    Contributor

    Change https://go.dev/cl/633395 mentions this issue: internal/gaby: exclude go1.17_spec docs from crawling

  2. gopherbot commented on Dec 13, 2024

    @gopherbot
    Contributor

    Change https://go.dev/cl/635176 mentions this issue: internal/devtools/cmd/rmdoc: delete crawled pages from corpus

  3. hyangah commented on Jan 3, 2025

    @hyangah
    ContributorAuthor

    Applied deletion:

    $ cd internal/devtools/cmd/rmdoc
    $ go run . -project oscar-go-1 -firestoredb prod https://go.dev/doc/go1.17_spec
    $ go run . -project oscar-go-1 -firestoredb prod https://go.dev/doc/go1.17_spec.html 
    

    Verified they don't appear in the similar doc search page.

    The rmdoc is a bit cumbersome to use since Gaby stores each section as a separate doc, the spec doc has many sections, and the rmdoc requires to approve deletion for each doc.

    Closing - we will use manual deletion as we find the obsolete or unsuitable docs.
    We don't have a good solution to automatically delete/remove obsolete pages yet, but we also don't have a plan to address it until it becomes a frequent issue.

  4. self-assigned this
    on Jan 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions