IMPORTANT: LiVID/Magnetik is no longer developed ! Using it in a production environment is strongly unwise and difficult.
The new project is called Magnyaa. Don't waste more time on this page
release 5 - “You Wouldn't Steal A Lion”
release 4 - “WTF Am I Reading”
release 3 - “Pissing Code Like I Mean It”
release 2 - “Nyaa~”
release 1 - “Do What You Want 'Cause A Pirate Is Free”
In the 23th, February 2012, some time after Megaupload's shutdown, many concerns appeared about The Pirate Bay (for example, DNS censoring in UK). While the P2P structure allows the data to remain even if the website is shut down, the BitTorrent technology is lacking a real search function. BitTorrent should be as decentralized as possible, and therefore the usage of .torrent files should be avoided (the connections being established with the DHT).
While better solutions are integrated in the BitTorrent clients (like the gossip-protocol search of Tribler), one can simply host a “little Pirate Bay” that, in combination with the search engines, should provide many many different sources from where getting the most important information about a torrent (it's name, btih and size - well we can add the seed- and leechcount, giving a hint of the torrent's popularity).
I started working with a full-PHP system, the torrent data being stored in infohash-named PHP files. Well it IS working, but created thousands of little files in the db/ subfolder. Manipulating a real database ? Well ok, but it shouldn't rely on a heavyweight like MySQL or PostGreSQL (not very easy to backup and restore). So I started messing around with SQLite3 and fSQL. And here we are :)
SQLite and fSQL are working well and all, but for very heavy databases they just aren't fast enough. We're talking about retrieving little data and parsing mid-long listings: it's much more efficient to split the data and load it on-demand as close as possible from the target. This leads us to Magnesia. We talk about it later..
Better check out before you get started ! My secondary host supports both SQLite2 and SQLite3, it didn't helped me when I wrote the queries. Hopefully it's 100% SQLite3 and has maximal compatibility. Take a look at phpinfo() if SQLite3 is enabled on your hosting. If something's wrong, please report and suggest fixes
fSQL, while supporting MySQL-like commands, is still a PHP script. Having PHP5 installed is much all you need
As said: for heavy databases we need something faster, much faster, but still portable. For Magnesia my choice is a plain file storage, in PHP files, the data being stored and loaded as arrays with a requiring as needed/asked. So no worry about millions of entries, PHP only loads the needed files and doesn't have to parse more than 1000 arrays. And it does it quite well and fast, on both PHP4 and PHP5. Simply said: it works fucking good on almost every web hosting for a potential heavy database (according to my calculations, the full The Pirate Bay database wouldn't weigh more than 2 GB and any data from it could still be called very fast).
Different releases work differently. The most reliable way to set up things right is to read the code Some releases feature the bash scripts. Magnesia's database is solely built with the bash scripts.
- customize “config.php” (you may want to change the title, description, the CSS, the table structure… and the addition password !) - upload - begin populating your database
→ make sure the data is correctly formatted before “wgetting” it to the instance ! The data retrieval can be affected by changes, especially on NyaaTorrents: maybe you'll have to modify the awk functions to get the right data out of the page.
This software is provided “as-is”, without any warranties. The author of this software doesn't encourage counterfeit content distribution. The author cannot be held liable of any misuse of this software. The code is neutral, only a human can do illegal things with it. It is the user's sole responsibility for downloading and sharing illegal content. The information returned by the database can be inaccurate, obsolete or plain fake: automated processes can't always succeed. To comply with some countries' laws, the hostmaster can be asked to remove manually any information about an obviously illegal torrent in the database.
Magnetik Nyaatorrents, published under Creative Commons by-sa by Mitsu.
* included: torrent-rw, published under GNU GPL by Adrien Gibrat: https://github.com/adriengibrat/torrent-rw/
* included: fSQL, published under GNU LGPL by Steve “the_unknown”: https://sourceforge.net/projects/fsql/
Bug report, feature requests, love declarations ? Head over here: