After two weeks of re-writing the filtering engine 3 times, I’m finally back to where I started. Yay!

Well, a little better than where I started. The more filters I added, the more complex and ugly the code to make them work became. After two re-writes, I’m finally happy with the 3rd iteration. It’s pretty modular and will allow me to easily add and tune additional filters later.

Other than that, what’s new since the last update?

  1. Can now filter (or hide entirely) posts and comments if they are below a set score. This can be done either globally or a per-community group basis.
  2. I ripped out the user tagging feature and am going to re-write it closer to how the community groups are done. The main reason is the original code was just a test/proof-of-concept. Additionally, once I do this, it would also allow creating filter policies that apply to user tags the same way they apply to community groups.
  3. Instead of only showing the first policy hit, all policies are evaluated, and each reason is shown. I wanted to avoid this, but since different policies/options can have different effects (hide vs filter), it was possible to “short-circuit” a more strict filter if it hits a less strict one first. Additionally, it is nice to see all the reasons a post was filtered rather than just the first hit.
  4. Editing the filter preferences is MUCH less of a fustercluck now. The global policy and the policies attached to community groups are now standardized, and there is now a standardized policy edit component (seen in post image). Not only is it more organized, it’s much more modular and allows editing filters from the quick settings menu.
  5. The policy name in the stub/placeholder (e.g. [Default Policy]) is clickable and will bring up a modal to edit that policy.
  6. The global filters that apply to everything are now called “Default Policy”. Currently, the names of the community groups that have filtering enabled are also the policy names. User tagging will likely be similar once I get that re-implemented.
  7. I’m adding user exceptions to the policies (work in progress). That should allow you to exempt specific users (or user regex patterns) from the policy (global or community-group based) while filtering or hiding everything else that the policy flags.
  8. I removed the “Help Farmer” filter option. I don’t want to have to maintain a list of “ask” communities, and you can achieve the same effect by putting those communities into a group yourself, enabling filtering for it, and setting a minimum account age to filter.

As you can see, the biggest areas of work in this version are to the filtering options.

Wish I had more to report, but re-writing the whole filtering engine was a huge grind, and I had to take several days off to avoid burning out completely. Now that that’s over with, I’m hoping the development pace picks back up on the remaining 45 items on my “to do” list for this release.

Ideas I’m Toying With. Thoughts?

  1. Option to hide post/comment scores until after you’ve voted. Basically to coach you to vote based on the content rather than joining bandwagon.
  2. Provisions to assign filter policies to user tags. Seems like it could be useful, but trying to think if that would do anything the existing global and community group-based filters don’t already do.
  3. ???

Screenshots

Please do not take anything personally if you show up in the screenshots of my filtering preferences. These were either chosen at random because they were at the top of the feed at the time of testing or were chosen because they have both Lemmy and Piefed versions/alts and were used to test wildcard and regex based filtering.

Also, all of the filtered items can be optionally hidden completely. They’re shown as the placeholder stubs here since that is more useful for a screenshot.

Filtered Post Placeholders

  • The option to filter low score posts is enabled with a threshold score of -5.

  • Ozma is filtered in my “News and Politics” group as well as anything with the keyword ‘trump’
  • I filter communities from feddit.org since I don’t speak German.
  • I don’t want to see slop from Mastodon in my feed

Misc Filtering Options for the Policy

Filter Users Directly or by Regex Pattern

Filter Communities Directly, By Regex, or by Instance

Filter Specific Keywords

Filter by Instance/Platform

Filter by Post’s URL Domain

Exempt Specific Users or User Regex Patterns from the Policy

Note: This is still work-in-progress

  • jet@hackertalks.com
    link
    fedilink
    English
    arrow-up
    2
    ·
    15 days ago

    I’ve thought about it some more; I think this would be a fairly useful compromise:

    A post is auto-hidden if it has more then (-5 default) votes, and less then (+3 default) votes

    This would leave controversial things up that at least have some backers, but universally hated things like illegal stuff, or spam, would be one sided and hidden.

    • Admiral PatrickOPM
      link
      fedilink
      English
      arrow-up
      2
      ·
      edit-2
      14 days ago

      +3 default? Not sure what you mean with that.

      Do you mean, for example, if the score is -5 but it has more than 3 upvotes, then show it?

      Right now, it only hides with negative total scores below the user-defined threshold and only if the user enables that filter option (disabled by default).

      I did tweak it a bit, though, since this post. If blind voting is enabled, the low-score filter is disabled.

      Also, I added blind voting and am liking that. I may add a config variable for instance admins if they want to make blind voting mandatory in their deployments (it’s disabled by default and user-toggleable otherwise)

      • jet@hackertalks.com
        link
        fedilink
        English
        arrow-up
        1
        ·
        14 days ago

        Do you mean, for example, if the score is -5 but it has more than 3 upvotes, then show it?

        Yes, exactly, that is my proposal.

        • Admiral PatrickOPM
          link
          fedilink
          English
          arrow-up
          2
          ·
          edit-2
          14 days ago

          I can do that. Might use a percentage rather than a fixed value, though. I’ll look into it and try to work that into its behavior.

          I should point out that the filter doesn’t kick in if it has 5 downvotes. e.g. A post can have 100 upvotes and 105 downvotes. The net score would then be -5 which would trigger the filter (assuming the threshold is configured to -5). If other people upvote it to bring its overall score up, it’s not filtered the next time it shows up.

          Does that change the proposal at all?

            • Admiral PatrickOPM
              link
              fedilink
              English
              arrow-up
              2
              ·
              14 days ago

              So might be better to not use a fixed threshold at all and just set a minimum upvote threshold with, maybe, an internal minimum score (-5?) to act as a guardrail?

              I’m basically trying to satisfy two crowds here:

              • One who doesn’t want to see content that’s been downvoted to oblivion (some rightly so, and others simply for deviating from the groupthink here)
              • One that posts stuff that is apparently controversial here and doesn’t want to be unnecessarily filtered (niche communities, etc).
              • jet@hackertalks.com
                link
                fedilink
                English
                arrow-up
                2
                ·
                14 days ago

                Fair enough, giving power to the users is good, my only concern is any default behavior