Gitlab-Issue-Listing-Script v0.5

Version: 0.5

Project Info

Gitlab Issue Listing Script (GILS) is a php front-end to the Gitlab API. It's intended as a means to work around Gitlabs awful SEO

Much like my original JIRA implementation (JILS) it provides a simple HTML view of information - it's intended as a read-only interface rather than one accepting input.

Read more…

tor-daemon telegraf plugin v0.2

Version: 0.2

Project Info

The tor-daemon plugin is an exec plugin for Telegraf allowing statistics to be captured from the Tor daemon on Onion services and relays.

Details on usage can be found at Monitoring the Tor daemon with Telegraf.

Release Notes

Version 0.1 implements the basic functionality of the plugin

Release Revision

This version was released in commit c30a1bd

Release issue tracking

Issues assigned to this release can be viewed in GILS

Plugin

The plugin can be downloaded from Github or from here.

tor-daemon telegraf plugin v0.1

Version: 0.1

Project Info

The tor-daemon plugin is an exec plugin for Telegraf allowing statistics to be captured from the Tor daemon on Onion services and relays.

Details on usage can be found at Monitoring the Tor daemon with Telegraf.

Release Notes

Version 0.1 implements the basic functionality of the plugin

Release Revision

This version was released in commit 8b60035

Release issue tracking

Issues assigned to this release can be viewed in GILS

Plugin

The plugin can be downloaded from Github or from here.

Gitlab-Issue-Listing-Script v0.4

Version: 0.4

Project Info

Gitlab Issue Listing Script (GILS) is a php front-end to the Gitlab API. It's intended as a means to work around Gitlabs awful SEO

Much like my original JIRA implementation (JILS) it provides a simple HTML view of information - it's intended as a read-only interface rather than one accepting input.

Read more…

RemoveAMP: V1.6

Project Info

RemoveAMP is a userscript for Greasemonkey/Tampermonkey designed to pull in a short snippet of javascript that will attempt to detect Accelerated Mobile Pages when they load, and navigate away from them to the fully functional canonical URLs. If the publisher hasn't specified the canonical, a DuckDuckGo search link will be injected into the page to help you find it.

See the Project README and FKAMP-1 for more information on why I consider this desirable.

As of April 2022, issue tracking can be found here

Read more…

Gitlab-Issue-Listing-Script v0.3

Version: 0.3

Project Info

Gitlab Issue Listing Script (GILS) is a php front-end to the Gitlab API. It's intended as a means to work around Gitlabs awful SEO

Much like my original JIRA implementation (JILS) it provides a simple HTML view of information - it's intended as a read-only interface rather than one accepting input.

Read more…

Gitlab-Issue-Listing-Script v0.2

Version: 0.3

Project Info

Gitlab Issue Listing Script (GILS) is a php front-end to the Gitlab API. It's intended as a means to work around Gitlabs awful SEO

Much like my original JIRA implementation (JILS) it provides a simple HTML view of information - it's intended as a read-only interface rather than one accepting input.

Read more…

RemoveAMP: V1.5

Project Info

RemoveAMP is a userscript for Greasemonkey/Tampermonkey designed to pull in a short snippet of javascript that will attempt to detect Accelerated Mobile Pages when they load, and navigate away from them to the fully functional canonical URLs. If the publisher hasn't specified the canonical, a DuckDuckGo search link will be injected into the page to help you find it.

See the Project README and FKAMP-1 for more information on why I consider this desirable.

Read more…

BUGGER Project Listing Plugin

(updated )

This content was originally published on benscomputer.no-ip.org

The script listed here is a 'plugin' allowing external applications to utilise the Project Table from within BUGGER. It requests details of all projects with 'Advertise' set to Y. It can be used as part of an SSI, or as a module in another app. The code is licensed under the GNU GPL V3 so feel free to use it as you see fit.



Download - MD5 Checksum: b217ff2dd085d95f67670783f72799dc

The script does not utilise caching at the moment, the response time appears to be pretty good so the additional cycles used to check caches would only lead to slower responses at the moment. Caching may be implemented in later releases however.

The next release of BUGGER will contain a project listing page, however the BUGGER page will list all projects stored within the database. This one simply shows projects that you have elected to allow 3rd party apps to advertise.

BUGGER: V0.21

(updated )

This content was originally published to benscomputer.no-ip.org



It's been a long time coming, but tonight a full build of BUGGER has been released. Although still sitting well within the realm of experimental software the 8250 lines of code in the latest release implements the following features;


  • Bug Reports
  • Project Summaries
  • Documentation Pages (Currently incomplete)
  • Release Pages (Currently incomplete)
  • TODO lists
  • Changelogs
  • User Access Control
  • Ability to Delete/Disable Users
  • Ability to Comment on Bug Reports
  • Admin Controlled Captchas for Bug Reports and Comments
  • Caching of Changelogs, TODO lists and Bug Report Pages
  • Administrators can refresh/delete cache files
  • Administrators can clear temporary files
  • Ability to mark bugs as Unassigned, Closed - Resolved, Closed - Need more info, Assigned to Developer, Suspended for testing.
  • Reporting of Comment Spam to Administrators
  • Timestamp logging of Releases/Documents/Comments
  • Autocompletion of name/e-mail address for Logged In users
  • 3 Levels of User Access Privilege - User, Moderator, Developer, Administrator

Pre-Requisites

To run BUGGER you will need;

A Webserver (Apache recommended)
Claims_DB
Claims_DB_listener
SendEmail
Awk, Sed, Grep
A text editor for initial configuration

The system will run without SendEmail, you just won't be notified of any new bugs that have been logged. It's strongly recommended that if you don't currently have SendEmai installed and can live without the alerts for a while, wait for the next release when there's less chance of your inbox getting filled daily!


Getting BUGGER V0.21


You can view the project changelog here.

You can download the release from here - MD5 Hash 8bddf67174834f679326b484a1d37c6e.

The Project Summary page is available here.

Installation instructions are included in the tarball, but you can also view them here.

The software still lacks a few features, however the next few months of development have been planned, and the project is already ahead of schedule! If you experience any issues with this release your feedback would be greatly appreciated, you can log reports in BUGGER, or if that should fail then please Contact Me directly.

ClaimsDB Listener: V0.23

(updated )

This content was originally published to benscomputer.no-ip.org


Claims_DB_listener V0.23 is now available. There's a single new feature, but it's so useful for improving the performance of front-ends I decided it was worthy of a release.

  • Table Checksumming

This feature has been available in Claims_DB for a little while, and was just bypassed by the listener. Using checksumming can allow a front end to decide whether or not to use a cached version. Whilst the queries themselves are usually quite quick, processing the returned data can take a little longer.




BUGGER: V0.1

(updated )

These release notes were previously published to benscomputer.no-ip.org


BUGGER has been online for quite some time, but the source is now available. It's still in the very early stages of development so unless you are sado-masochistic it's probably not worth installing quite yet.

The basic building blocks are there, but it does lack some basic features (Authentication protocols being one of those). You can use it to report and track bugs, but you'll need to get down and dirty in the scripts as portability has not yet been written into it.

Eventually it'll be a breeze to install and maintain, it's just not quite there yet!

You'll need

Claims_DB
Claims_DB_listener

as well as the basic BusyBox apps - Awk, Sed etc.

For more information view the Readme (also included in the tarball)

Download BUGGER V0.1 here.

ClaimsDB Listener: V0.22

(updated )

These release notes were originally published to benscomputer.no-ip.org

V0.22 of Claims_DB_listener has a small number of changes;

  • Re-nicing of database backups
  • Support for HTTP POST Requests

There are also a number of bug-fixes. No new functionality has been added to Claims_DB itself in a while, which has limited the changes that can be made in this project. As the project develops more features will be added.

You can file bug reports in BUGGER

You can view the Claims_DB_listener changelog here.


Getting the latest version

You can download the protocol documents from here.

You can download the script from here.


Previous Version

Version 0.21 of Claims_DB_listener is available from here.

ClaimsDB: V0.21

(updated )

This content was originally published to benscomputer.no-ip.org


The only updates in release V0.22 are in the script read_records_claims.sh. You can view the changelog here.

Updates in this release are;

  • Ability to clear the temporary directory of any leftover files
  • The ability to return a count of all records in either the whole database or in the specified table.
  • General bug fixes ( 1, 23, 19, 29 )


The updated file can be found here.

The updated Protocol Documentation can be located here.

New Bug Reporting System online

(updated )

This content was originally published to benscomputer.no-ip.org

The original system was CGI based and actually looked quite nice. The bastardisation of the interface seen in today's static mirror came much later, after it was rewritten in PHP


Although there's not been many updates posted on the site recently, there has been a lot of work going on in the background. Although there are a number of projects available for download on the site, to date there's been no easy way to report Bugs and errrors.

This has now been rectified in the form of BUGGER. The functionality is limited at the moment, you need to know the Bug ID in order to load information on it, but improvements are planned in the very near future. The system is currently taking precedence over Claims_DB as there is obviously very little point in further development until we can receive some feedback on the system.

The name is simple to understand, it's BUGGER as in - The systems broken! Have you BUGGERed it?

If anyone wants to take a look at the various interfaces, there is a bug currently reported in the system with Bug ID 1. You can also access it via this link. BUGGER will have the Captcha Project integrated into it's interface very soon, but for the time being there is a basic Captcha style test.

ClaimsDB Listener: V0.2

(updated )

This content was originally published on benscomputer.no-ip.org

To coincide with the new functionality of Claims_DB a new version of the Claims_DB_listener script is available. You can grab it from here.

There are of course some new METHOD's available to serve your requests, but the previous functionality remains unchanged. On the TODO list is seeing if the speed of certain transactions can be sped up by means of caching. This is probably quite a way off given the work involved in making sure that the script only every uses a cached version when the Database response will not have changed.

In order to learn about the new METHODs you'll probably want to read the updated Protocol Docs. You can also view the changelog.

ClaimsDB: v0.2

(updated )

This content was originally published on benscomputer.no-ip.org

As promised, Claims_DB V0.2 is officially released, there have been a few changes in the latest development. These include;

  • Implementation of new functions
  • Prevention of a Data Clashes in Multiuser systems
  • Activation of record deletion function

The system still uses temporary files but uses its PID to try and make the filename unique. It's still not fantastic at tidying up behind it, but it's getting there. If you want a faster response, it may be worth configuring the system to use a RAM disk as its temporary location.

The temporary file location is now defined in /etc/claims_db.conf allowing for quick configuration throughout the system.

There are also some minor bugfixes, see the changelog for more details.

This is an incremental release so please replace the components with the ones listed here, if you have not yet installed the system, please grab the V0.1 tarball and replace its program files with these.

A full release will be made at the next release stage!

As ever, any bug reports are welcome. Please leave them on the Contact Me Page

Files

CLAIMSROOT/bin/read_records_claims.sh
CLAIMSROOT/bin/insert_record_claims.sh
/etc/claims_db.conf
CLAIMSROOT/Docs/Protocol_Documentation


ClaimsDB Listener: V0.1

(updated )

This content was originally published to benscomputer.no-ip.org


As promised previously, Claims_DB has become net aware. Technically it's still not net aware, but I've created a 'listener' script for it. The basic impact of this is that you will be able to host your database on a seperate system to the front-end.

The listener script is actually a CGI script, so you will need a HTTP daemon running on the database server, but with the likes of LightHTTPD using so little memory, it felt a bit pointless writing an actual daemon. There is an argument that running a HTTP daemon on the server un-necessarily provides another attack vector and increases overhead, but it's apples and pears.

I lack the time that I'd want to ensure that any listening network daemon was secure, so it seems better to allow the end-user to choose which service they want to run. As long as the HTTP server supports CGI's and you have BASH on the system, this script should work for you.

You can grab V0.1 of the script here.

To understand how to serve the system requests, take a look at the Protocol Documentation

It's being run as a seperate project so that I can focus on getting the functionality of Claims_DB up to scratch, it's likely that the listener script will lag behind the main project a little most of the time (although it does currently support functionality that I haven't finished implementing yet!)

ClaimsDB: 0.1b

(updated )

This content was originally published on benscomputer.no-ip.org

Current Release: 0.1B

Claims DB is a simple flat file database engine written in BASH. It supports the addition, deletion and basic querying of records.  All it's components are released under the GNU GPL.

All written Documentation is released under the Creative Commons Attribution-NonCommercial-ShareAlike2.5 License.

For those wanting to skip the background of why this program was created, and a few other things. Click Here to skip to Getting Claims_DB


Why write it in BASH?

To see if I could. It's as simple as that


Why such limited functionality?

Currently, the engine only supports the querying of one column, any further querying has to take place within the client. However, more advanced querying should be implemented in later releases.


How do I use it?

This release comes with a basic CLI client. At the moment, adding a new database does require some manual work, especially if you want to add Queries, Reports and Forms. However there are a couple of example databases within the client, so that you can see how to create Queries, Forms and Reports. The procedure is a little more complex than I would like, but I am intending to simplify it with either a GUI or a Web based GUI.


How does the system reference Queries, Forms and Reports?

Rather than having the system check the Queries folder for all Queries in there, the system has its own built in database, which contains details of all Queries, Forms, Reports, Tables and even all columns within each Table. This means that you can effectively hide Tables and Queries from the end user. So if a query is written specifically for a report, and is otherwise useless, you don't need to have it listed under the queries option.

Although the system does keep a list of Column names for each table, this functionality isn't really used yet. It is however a useful resource to keep up to date, as when you create a Form/Report/Query it's useful to have a print to remind you which Column is which.


How is the Data stored?

The tables used are simply CSV files. You can take the straight from the database into OpenOffice, Excel or whatever. It also means you can also export data from these programs to the database (though there is still some manual fingerwork to do at this point)


Why Implement this when there are so many other DBMS solutions around?

Pretty much for the same reason I wrote it in BASH, for a challenge. It also means I should be able to get the system to do exactly what I want it to. For example, I want my data stored in CSV format for easy reference, but most DBMS Programs won't let you create Relationships between tables in this format. There's a very good reason for it, and this system is not truly relational. BUT, I can make it act as though it is. For example, if you load the Bike_Parts Example Database, and try to add an item to stock using the form, it will ask you for the primary key of the part you want to add. It then queries the other table to retrieve the details for you.

Not impossible to do in other systems, but combined with the other planned benefits, for my current needs this system should hopefully be best.


You Mentioned a GUI/Web Based GUI, Whats planned for this?

Obviously the basic features such as adding Data, Retrieving data whether through Query or otherwise, reports and forms. Eventually, the GUI/WGUI should be a far more full DBMS than the CLI could ever really be. The CLI is intended for basic transactions, but there is probably only so far it can go. The idea however is to be able to port code between each interface, for example the basic requests such as Querying should be able to be copied across almost verbatim.

For those functions, it is largely a case of adapting how the Interface reads input, and how it displays output.

That said, the GUI/WGUI will eventually have a query builder. I'm not quite sure how I am going to implement it yet, but the intention is to allow the user to specify (and hopefully save) the columns and criteria.


So the system is going to get harder to use?

No, quite the contrary really. The idea is that it will get easier to use, I'm eventually aiming for a reasonably simple user experience. Aside from the install, I don't want the user to have to open a text editor at any stage, but that's some way off as I'm currently more interested in implementing the basic functionality before I look too much at simplifying it.

You also have to expect that in the reasonable future at least, anyone that uses the system is likely to be a cut above Joe Average. The system runs on an OS that not all that many people use. Once Microsoft drive more of their users away, this could change very quickly, but by that point the experience should be a bit easier.


Getting Claims_DB


The system is available from this Gzipped Tar file.


311a62c959051c4d52351d081decf395 Claims_DB.tar.gz

For those who just want to browse the code, you can also look at the main components here. None of the System Database tables are included here though, so you may want to grab the tar file so you can see what is being referenced.


Database Engine

insert_record_claims.sh
read_records.claims.sh
db_backup.sh



Interface

Claims_DB_CLI.sh



Development Documentation

This is the documentation that I created as I wrote the system. It is not a lot of use for teaching you how use the CLI, but the Protocol Documentation will show you how calls to the Database Engine are made.

Planning
Testing
Protocol Documentation (slightly out of date, doesn't include the -query-line call)



Config Files

An example /etc/claims_db.conf
config for the Database Engine
Config for the CLI







Sitemap Generation Tools

(updated )

This content was originally published on benscomputer.no-ip.org

This is the main project page for the Sitemap Generation tools,

All code and information is released under the GNU GPL and is copyright Ben Tasker 2009, however the Howto does refer to Google Sitemap Generator, which is a utility not written or owned by me. It is freely available under the BSD 2.0 License.

For a copy of the GNU GPL please either look within the downloads


The tools are designed to make creating and managing sitemaps far easier on a Linux server, if you use a Content Management System then it probably already generates sitemaps for you, if it doesn't then these tools can ssave you from doing it manually, or being tied to the 500 page limit that many of the utilities have.

There are a range of Dependencies, but most should already be present. You will need

  • BASH
  • wget
  • tr
  • awk
  • sed
  • grep


    • As ever, the software is made freely available with absolutely NO WARRANTY, it does no harm on my system and shouldn't on you, but I make no claim that it definitely wont. All software is installed and run at your own risk.

    The software consists of three main sections, each can be run independantly, or from within the wrapper (that I haven't yet written, so check back!)
    All the software is designed to be run from a cron job, so set it on a convenient schedule.

    UPDATE: I've discovered a slight issue with the URL List Generation script, for some reason wget can be a bit picky about the use of --spider, if your URL List only contains one line after running the script try configuring and running this script instead. It retrieves the pages and deletes them afterwards, so this will affect the amount on Bandwidth used. Obviously if you are running the script from the target server this will have little impact.

    For information on howto setup and configure the Google Sitemap Generator to run automatically please read this HOWTO (PDF)

    To generate HTML sitemaps please install, configure and run HTML Sitemap Gen V0.1. Installation instructions are contained within the README file.
    The program will generated a HTML sitemap, seperated by file type, HTML is listed first (including PHP etc) followed by PHP, Images, Plain Text Files and Finally all other Document types.
    The Layout of the finished page is defined in a template, the script simply insert the information, for an example of the output please visit my sitemap

    HTML_Gen_V0.1.tar.gz README CODE
    MD5 Checksum
    086b710d31de1412d4b93be7b5db6455 HTML_sitemap_gen_V0.1.tar.gz


    To Generate the URLLIST upon which the above two programs rely (and Yahoo! uses) please install, configure and run URL List Generator V0.1. Installation instructions are included within the README file

    URL_List_gen_v0.1.tar.gz README CODE
    MD5 Checksum
    ca9fe330ea1b202665e0abe6aedfb87c URL_List_gen_v0.1.tar.gz


    If utilising all three programs in cron jobs, it is worth noting that there is no specific order in which they should be called. It seems logical though that there is very little benefit in calling a program that relies on an URLLIST until the list itself has been refreshed. So most people would call the URLList Gen program first. Whilst it is unlikely that Google would ever crawl your site immediately after receiving a ping, on the off chance, it would be wise to wait until the HTML sitemap is created before running the Google Sitemap Generator.

    Be aware that several of the scripts do post a lot to STD:OUT, so your Cron log may fill up quite quickly, the wrapper script will give you the option to specify a seperate log file to use, and will also call the programs in the most logical order.

    The Generator Wrapper is intended to be called from a Cron job (or the command line) and simply calls the programs in the most logical order. It also reduces the amount pushed to STD:OUT, you can either specify a log file or most output will be sent to /dev/null.

    Usage is as follows;

    sitemap_wrapper.sh -h # Prints a short Synopsis
    sitemap_wrapper.sh # Send all log information to /dev/null
    sitemap_wrapper /path/to/some/file # Sends all logging information to the specified file.

    Sitemap Wrapper

    Netmanage V0.1

    (updated )

    This content was originally published on benscomputer.no-ip.org

    NetManage V0.1 is a simple GUI driven system to allow you to easily access Web Configuration Interfaces for Hosts on your network. It is especially useful where you have Interfaces listening on non standard ports, once the details are entered into the Clients configuration file, you simply need to select that host from the menu.

    The only real dependencies are BASH and Kdialog.

    For installation instructions read the README file

    Download

    Checksum

    1050a133bcab329aa118dc3c7def82a7 NetManage-0.1.tar.gz