• Allowing your internal search engine to index Gitlab Issues, Commits and Wiki pages

    I've basically lived my life inside issue tracking systems - it used to be JIRA, and I had built tooling allowing effective indexing/mirroring of JIRA issues, but then Atlassian decided to all but do away with on-prem users. So, like many others, I moved over to Gitlab instead.

    As a product, Gitlab's great: it's got project lifecycle management, issue tracking, wiki support and even a container registry baked straight into it.

    However, it's indexability is basically non existent - even if you have projects marked as public, significant parts of pages are fetched by javascript, so a crawler can't index things like issue references (no, I'm not kidding) unless they execute javascript. It's like they reverse search engine optimised...

    It's not just on-prem Gitlab either, it also affects their cloud offering.

    Search within Gitlab is excellent, but this is of little use if you have things spread across systems and want to search from a single central point (to be fair to Gitlab, their idea is that everything you do should be within their solution, but life rarely actually works that way).

    This documentation details how to use my new tooling (rudimentary as it currently is) to expose an indexable version of your Gitlab life. Although this is focused on an on-prem install, this tool should work with their cloud offering too, as the APIs are the same (but I haven't tested against it).

  • Allowing your Internal Search Engine to Index JIRA Issues

    I use a number of tools on my network, including a private JIRA install (i.e. you need to log in to view anything) and the Sphider PHP search engine (I've generated a lot of documentation over the years).

    Unfortunately the two aren't exactly compatible, as Sphider has no way to log into JIRA, but I wanted my JIRA issues and comments to be indexed so that relevant items can be included in my search results. One option would be to set JIRA to public mode, but I'd rather maintain the need to log in.

    So instead I created a simple PHP script - JIRA Issue Listing - to generate a list that Sphider could index, but would redirect 'real' users to the relevant issue on JIRA.

    This post is the documentation for that script

  • Implementing Secure Password Storage with PHPCredlocker and a Raspberry Pi

    Password storage can be a sensitive business, but no matter whether you're using PHPCredlocker or KeePassX, dedicated hardware is best. The more isolated your password storage solution, the less likely it is that unauthorised access can be obtained.

    Of course, dedicated hardware can quickly become expensive. Whilst it might be ideal in terms of security, who can afford to Colo a server just to store their passwords? A VPS is a trade-off - anyone with access to the hypervisor could potentially grab your encryption keys from memory (or the back-end storage).

    To try and reduce the cost, whilst maintaining the security ideal of having dedicated hardware, I set out to get PHPCredlocker running on a Raspberry Pi.

    This documentation details how to build the system, a Raspberry Pi Model B+ was used, but the B should be fine too

  • Unable to check for Euro Symbol in POST data

    I came across an interesting issue this week, having created a form to submit data I then needed to check against stored values to pre-populate fields if a user had already completed the form. Pretty simple stuff really, not much more to it than

    ?>
    <input type="checkbox" value="$val" <?php if ($stored == $val){ echo " checked"; } ?>
    >

    But, I found that where the euro symbol (€) is concerned, things can get quite difficult. 

    This documentation details the issue I found and how to work around it.