Merge pull request #516 from timwhite/higher-order-custom-filter

Allow semi easy addition to filtering logic

docs/custom-filters.md

@@ -0,0 +1,41 @@
+# TubeSync
+
+## Advanced usage guide - Writing Custom Filters
+
+Tubesync provides ways to filter media based on age, title string, and 
+duration. This is sufficient for most use cases, but there more complicated
+use cases that can't easily be anticipated. Custom filters allow you to 
+write some Python code to easily add your own logic into the filtering.
+
+Any call to an external API, or that requires access the metadata of the
+media item, will be much slower than the checks for title/age/duration. So
+this custom filter is only called if the other checks have already passed.
+You should also be aware that external API calls will significantly slow
+down the check process, and for large channels or databases this could be
+an issue.
+
+### How to use
+1. Copy `tubesync/sync/overrides/custom_filter.py` to your local computer
+2. Make your code changes to the `filter_custom` function in that file. Simply return `True` to skip downloading the item, and `False` to allow it to download
+3. Override `tubesync/sync/overrides/custom_filter.py` in your docker container.
+
+#### Docker run
+Include `-v /some/directory/tubesync-overrides:/app/sync/overrides` in your docker run
+command, pointing to the location of your override file.
+
+#### Docker Compose
+Include a volume line pointing to the location of your override file.
+e.g.
+```yaml
+services:
+  tubesync:
+    image: ghcr.io/meeb/tubesync:latest
+    container_name: tubesync
+    restart: unless-stopped
+    ports:
+      - 4848:4848
+    volumes:
+      - /some/directory/tubesync-config:/config
+      - /some/directory/tubesync-downloads:/downloads
+      - /some/directory/tubesync-overrides:/app/sync/overrides
+```

tubesync/sync/filtering.py

@@ -6,6 +6,7 @@ from common.logger import log
 from .models import Media
 from datetime import datetime, timedelta
 from django.utils import timezone
+from .overrides.custom_filter import filter_custom
 
 
 # Check the filter conditions for instance, return is if the Skip property has changed so we can do other things
@@ -33,6 +34,11 @@ def filter_media(instance: Media):
     if filter_duration(instance):
         skip = True
 
+    # If we aren't already skipping the file, call our custom function that can be overridden
+    if not skip and filter_custom(instance):
+        log.info(f"Media: {instance.source} / {instance} has been skipped by Custom Filter")
+        skip = True
+
     # Check if skipping
     if instance.skip != skip:
         instance.skip = skip

tubesync/sync/overrides/custom_filter.py

@@ -0,0 +1,39 @@
+"""
+    This file can be overridden with a docker volume to allow specifying a custom filter function to call. This allows
+    for higher order filtering for those that really want advanced controls, without exposing the web interface to
+    potential RCE issues.
+
+    You are simply provided with an instance of Media, and need to return True to skip it, or False to allow it to be
+    downloaded.
+
+    To use this custom file, download this file and modify the function to do your check for skipping a media item.
+    Then use docker volumes to override /app/sync/overrides/ with your custom file (it must be called
+    `custom_filter.py`)
+    e.g. your `docker run` could have `-v /some/directory/tubesync-overrides:/app/sync/overrides`
+    or docker-compose could have
+        volumes:
+            - /some/directory/tubesync-overrides:/app/sync/overrides
+
+
+    The logic is that if any condition marks an item to be skipped, it will be skipped. To save resources, this
+    custom filter won't be called if any other filter as already marked it to be skipped
+"""
+
+from ..models import Media
+from common.logger import log
+
+
+def filter_custom(instance: Media) -> bool:
+    # Return True to skip, or False to allow the media item to be downloaded
+
+    # Put your conditional logic here
+    if False:
+        # It's in your best interest to log when skipping, so you can look at the logs and see why your media isn't
+        # downloading
+        log.info(
+            f"Media: {instance.source} / {instance} has met some custom condition. Marking to be skipped"
+        )
+        return True
+
+    # Return False if we aren't skipping the media
+    return False