this guide will detail the process I go through in updating and maintaining the kpg torrent. most of this is done using shell scripts specifically designed for the torrent, but the core scripts for encoding, tagging and renaming files could be used for a general music library as well. the remaining scripts could be used too with some small modifications.
the scripts are written in posix shell and have only been tested on linux but they should technically work on mac as well. each script has detailed comments throughout explaining how it works and what programs are required, though much of that will be reiterated in this guide too.
but I use windows! what do I do?
just install linux already...
...or if you must stick with winblows then ask an LLM to rewrite the scripts in python or powershell. they should be able to do that without issue. most of the heavy lifting is done by programs like metaflac and ffmpeg anyway, not anything that's linux exclusive.
life is so much better when you can automate things. don't spend your days doing things you don't have to. manually setting every single tag in a gui program like musicbee or foobar would take forever. existing automated options like musicbrainz / picard are also unreliable as they don't have consistent tagging standards and plenty of releases are tagged incorrectly or missing from their database.
basically all of my music is downloaded from jpopsuki which is a private tracker for asian media. the users there upload music in the best quality available and anything missing can be easily requested with a bot that automatically rips music from genie. other notable private trackers for music include red and orpheus, although I'm not in these and don't know how much kpop their libraries contain. soulseek is another option for p2p downloading which I tend to use as a last resort for old and obscure music that might not be on streaming services or private trackers.
the korean music streaming sites worth ripping from are bugs and genie and they both require a korean phone number or id to use. you'll also have to pay for a premium account to get high res file downloads. if you manage to make an account on either platform, then you can use the program rsack to rip music from them.
other sites that provide high res files are tidal and qobuz which can be ripped from using orpheusdl. modules and additional info regarding this can be found here.
FYI I've never actually used rsack or orpheusdl before and neither have been updated in quite some time so I can't guarantee that they still work perfectly.
I typically use r/kpop's "this week in kpop" posts to keep track of new releases. a small group of rym users also maintain a calendar every year. between the two you shouldn't miss out on anything new.
some general rules to follow:
I start by getting the torrent files from jpopsuki and placing them in four different directories.
when you load the torrent files into your client (I recommend qbittorrent) you can then download them into four separate directories as listed above.
once the files are all downloaded make a copy of them in a new directory. this copy will be used for tagging and adding to the library. the original files must be maintained for seeding purposes and their metadata can't be edited.
below is what my directory structure looks like when tagging new files. yours doesn't have to match this identically but visualising it like this may help to better understand some of my instructions later on. it's assumed all scripts will be run from inside scripts/.
kpg
├── lists
│ └── db.csv
├── logs
│ ├── foo.log
│ └── bar.log
├── new
│ ├── additions
│ │ └── album 1
│ ├── artists
│ │ └── album 2
│ ├── releases
│ │ └── album 3
│ └── upgrades
│ └── album 4
└── scripts
├── foo.sh
└── bar.sh
before running any scripts that modify files you will need to do a few things:
install the required programs and ensure they are in your $PATH
download all my scripts and place them in the scripts directory. make sure they are executable (chmod +x foo.sh).
if you already have a library of music that has been tagged correctly (eg. the kpg torrent), run list.sh -d /path/to/library -c to generate a fresh tag database. the database is placed under lists/db.csv and will be used to automate a significant portion of the tagging process. this database should be run once per library update.
move any loose files into a directory. eg. sometimes a single album might come downloaded only as song.flac rather than a directory containing the file and cover art. create a directory for each of these loose files and move them inside. something like artist - song/song.flac is fine, the name itself is not important.
delete any album sub directories. some torrents come with additional image/disc directories and these should be removed since they aren't needed. these can be found by running find new/ -mindepth 3 -type d. in the case of multi-disc albums, just move all tracks into the same directory.
prep.sh prepares files for tagging by:
flac or re-writing with ffmpeg (mp3)rsgainto use it, run prep.sh -d ../new -m
once the script is finished, check logs/prep.log for any errors. files with decoder errors will need to be replaced. these are uncommon but can show up once in a while.
flac.sh uses metaflac to set the following tags for flac files:
to use it, run flac.sh -d ../new -m
this script makes use of lists/db.csv to automate as much tagging as possible, but manual intervention is unavoidable especially when it comes to artist tags. if you can read hangul, it really helps to speed up manual input. when metadata is missing, I typically consult:
when the script is finished it will prompt you to remove any additional unnecessary tags, which you must do. this can also be done by running the script with the -x option.
tagging flacs is the most time consuming part of the entire updating process, so you may want to run this script on each individual directory (eg. new/additions, new/releases...) one at a time. you can also ctrl+c at any point to exit then re-run the script later to start where you left off.
mp3.sh uses eyeD3 to set the following ID3v2.4 tags / text frames for mp3 files:
to use it, run mp3.sh -d ../new -m
much of what was written for flac tagging also applies here.
this is currently unsupported, sorry.
rename.sh renames directories and files using the following format:
${artist} [${date}] ${album} [${codec}-${bitdepth}-${samplerate}]${discnumber}-${tracknumber} ${title}.${ext}to use it, first perform a dry run with rename.sh -d ../new. next, check logs/rename.log for any missing tags and resolve these issues using mf.sh. once the dry run produces no errors, you can modify files by running rename.sh -d ../new -m.
next, look through all the albums and use mf.sh to manually fix any title tags that use non-english characters or unnecessary phrases such as "the second album" or "1st digital single". finally, you can run rename.sh -d ../new -m one more time to rename directories after making manual tag edits.
cover.sh converts, downloads and embeds cover art using sacad, imagemagick, metaflac and eyeD3.
to use it, first run cover.sh -d ../new -c to convert any existing cover art and download missing cover art. next, check logs/cover.log to find any entries that are listed as 'missing cover art' and manually download these covers. open up all the cover art files in your image viewer and manually replace any that are incorrect or low quality. finally, run cover.sh -d ../new -e to embed cover art, then cover.sh -d ../new -x to remove all additional files.
3000x3000 is the standard resolution I aim for here, anything above that is unnecessarily large. many covers are only available in the 1400-2400 range though, which is still fine. older music from the early 2000's or before may be even lower res and in these cases avoid using upscaled art as it often looks worse than the original low res art.
high res cover art can be sourced mostly from bugs, but you will need to change the url after you open up the image in a new tab. this can be automated with an extension such as redirector. example url change:
https://image.bugsm.co.kr/album/images/50/xxx.jpg -> https://image.bugsm.co.kr/album/images/3000/xxx.jpg
other useful cover art resources are rym and yandex (for older albums).
mkmv.sh takes a master directory, scans it's subdirectories and creates a new directory for each unique artist found using the format: ${master}/${artist}/${quality}/Korean. it then moves all the releases for that artist inside the newly created directory.
to use it, run mkmv.sh -d ../new -m
please be aware that this only adds a korean language directory. other language directories and releases will need to be handled manually.
genre.sh uses metaflac and eyeD3 to set the genre tag for these specific release types:
to use it, run genre.sh -d ../new -m
for the script to work properly, albums must first be sorted into their appropriate language directories, as mentioned earlier.
check.sh runs several tests to ensure everything is set as per the library standards.
to use it, run check.sh -d ../new, then look for errors in logs/check.log.
this script does not do any modifications as there are too many variables to handle and too many places where it could go wrong. instead the script will just print a list of errors for you to handle manually.
once everything is tagged correctly, there are several small steps to go through in order to finalise the library update:
run changelog.sh -d ../new to generate a new changelog.html file you can use when updating the kpg website.
stop and remove the existing kpg torrent from your torrent client.
manually move all the new albums into the existing library location.
use mf.sh to make changes and tag edits to the existing library if necessary.
run stats.sh -d /path/to/library to generate the updated library stats.
create a new torrent file in your torrent client. leave the tracker list empty and disable private mode to ensure the torrent uses the dht network. once created, upload the torrent file to catbox.
update the website and git repo with the new changelog, index page, script edits, rss feed etc.
...and that's it, you're done! time to post the update on /t/.
this step is optional, but just to be safe the library should be backed up on a separate drive after each update. this is best done using an rsync command like so:
rsync -ha --del --info=progress2 /path/to/library /path/to/backup
