Improving Nextcloud's Thumbnail Response Time

Details

Category: Linux

Published: Friday, 01 November 2019 11:09

Written by Ben Tasker

I quite like Nextcloud as a self-hosted alternative to Dropbox - it works well for making documents and password databases available between machines.

Where photos are concerned, the functionality it includes offers a lot of promise but is unfortunately let down by a major failing - a logical yet somewhat insane approach to thumbnail/preview generation.

The result is that tools like "Photo Gallery" are rendered unusable due to excessively long load times. It gets particularly slow if you switch to a new client device with a different resolution to whatever you were using previously (even switching between the Android app and browser view can be problematic).

There's a Nextcloud App called previewgenerator which can help with this a little by pre-generating thumbnails (rather than waiting for a client to request them), but it's out of the box config doesn't help much if, like me, your photos are exposed via "External Storage" functionality rather than in your Nextcloud shared folder. Even when photos are in your shared folder, the app's out of the box config will result in high CPU usage and extremely high storage use.

This documentation details how to tweak/tune things to get image previews loading quickly. It assumes you've already installed and configured Nextcloud. All commands (and crons) should be run/created as the user that Nextcloud runs as.

Read more ...

Python3 - TypeError: encoding without a string argument

Details

Category: Development/Programming

Published: Saturday, 16 November 2019 12:47

Written by Ben Tasker

I thought I'd document this as although the cause/fix is fairly simple, searching for the error string encoding without a string argument gives a lot of hits for a similarly structured but different error - string argument without an encoding.

An example backtrace might be:

Traceback (most recent call last):
  File "./profiler.py", line 346, in 
    meta['config_files']['pdns'] = zip_and_compress(read_file_content('/etc/powerdns/pdns.conf'))
  File "./profiler.py", line 289, in zip_and_compress
    gz = gzip.compress(bytes(s,"utf-8"))
TypeError: encoding without a string argument

With the example code being fairly simple

def read_file_content(path):
    ''' Read the entirety of a file into a variable
    '''
    file_content = None
    with open(path, 'rb') as content_file:
        file_content = content_file.read()

    return file_content

def zip_and_compress(s):
    ''' Config files can get quite sizeable. To keep the size of our output DB down
    we gzip and then ascii armour them
    '''
    gz = gzip.compress(bytes(s,"utf-8"))
    return base64.b64encode(gz).decode("utf-8")


zip_and_compress(read_file_content('/etc/powerdns/pdns.conf'))

Read more ...

Disk automatically unmounts immediately after mounting

Details

Category: Linux

Published: Friday, 13 September 2019 14:39

Written by Ben Tasker

When it happens, it's incredibly frustrating - you've had a disk replaced on a linux box, the disk has shown up with a different name in /dev, so you edit /etc/fstab and then try to mount the disk.

The command runs, without error, but the disk isn't mounted, and doesn't appear in df

This documentation details the likely cause, and how to resolve it

Read more ...