Plugin For KoReader
The problem
I recently decided to get back to studying, both to improve my programming skills and to continue my personal development.
Some friends recommended books on psychology, philosophy, and other topics that caught my interest — and, of course, I kept adding everything to my Kindle.
My idea was simple: read on the Kindle, highlight the important parts, and automatically send those annotations to my Obsidian vault, where I keep all my studies and notes. That way I could easily revisit the most striking concepts from each book.
But then came the problem.
My plan was to use Readwise to capture the highlights and send them to Obsidian through a simple plugin.
To my unpleasant surprise, I discovered that Readwise doesn’t capture highlights from books manually added to the Kindle — only those purchased from the Amazon store.
And I knew the highlights existed, because they showed up normally when I opened the book in the official app.
The solution
I could have just given up on automation and kept copying everything manually.
Or I could have looked for some third-party tool to solve the problem.
But I decided to go another route: completely abandon Amazon’s closed ecosystem, jailbreak my Kindle, and create my own solution.
After some research on jailbreaking, I ended up finding an article about KOReader — an open source document reader for e-ink devices, with plugin support and extensive customization. Exactly what I was looking for.
With the Kindle soul freed, I installed KOReader and started looking for documentation on how to develop plugins.
But, to my surprise, there was no “getting started” — not even a search bar to find functions in the wiki.
In other words: if I wanted to make this work, I’d have to explore the source code and learn how the ancients did it.
The action plan
Before diving into the code, I decided to define the basic requirements for how this plugin should behave.
The basic idea is that the plugin should:
- Fetch the highlights from the currently open book;
- Read the saved content;
- Send it to an HTTP server that should handle it appropriately.
This way the plugin can communicate with any server that has a route to handle this.
For the visual part, we need:
- A form where the user can enter the IP and port where the server will be running;
- A button to trigger the sync event;
- And, of course, a debug button.
Getting hands dirty
As a starting point, I chose two native KOReader plugins as a base:
- The Calibre one, which would give me a foundation for how to display components on screen;
- And the Exporter, which is a plugin already designed to handle highlights.
The first task was to figure out how to start the plugin and show a simple “Hello world” on screen — and for that, the structure below was necessary:
1local InfoMessage = require("ui/widget/infomessage")
2local UIManager = require("ui/uimanager")
3local WidgetContainer = require("ui/widget/container/widgetcontainer")
4local _ = require("gettext")
5
6local ObsidianSync = WidgetContainer:extend({
7 name = "obsidiansync",
8 is_doc_only = false,
9})
10
11function ObsidianSync:init()
12 self.ui.menu:registerToMainMenu(self)
13end
14
15function ObsidianSync:addToMainMenu(menu_items)
16 menu_items.obsidiansync = {
17 text = _("Obsidian Sync"),
18 sorting_hint = "tools",
19 sub_item_table = {
20 {
21 text = _("Export Highlights"),
22 callback = function()
23 UIManager:show(InfoMessage:new({
24 text = _("Hello, plugin world"),
25 }))
26 end,
27 },
28 },
29 }
30end
31
32return ObsidianSyncRoughly explaining this code:
we initialized a class called ObsidianSync, which inherits from the WidgetContainer class all the basic methods and attributes we need to create a plugin/widget in KOReader.
We also created an init method, which will be called when the app starts and will be responsible for adding the plugin to the interface.
Finally, we created the addToMainMenu method which, as the name suggests, is responsible for adding the plugin’s visual elements to the menu.
Event to capture basic information
After creating the basic plugin structure, I needed to figure out how to capture the basic information from the book we’re working with, things like title, number of pages, reading progress, and, just as important, the highlights.
So, to start this part, I decided to create a debug method that would return the basic information about which book we’re on, what its extension is, and in which directory it’s located.
1function ObsidianSync:getFileNameAndExtension(path)
2 local info = {
3 dirname = path:match("(.+)/[^/]+$"),
4 filename = path:match("([^/]+)%."),
5 extension = path:match("%.([^%.]+)$"),
6 }
7
8 return info
9end
10
11function ObsidianSync:debug()
12 local info = {}
13
14 table.insert(info, "====== DEBUG INFO ======")
15 table.insert(info, "")
16
17 if not self.ui.document then
18 table.insert(info, "❌ Open a document to proced")
19 self:info(table.concat(info, "\n"))
20 return
21 end
22
23 local fileInfo = self:getFileNameAndExtension(self.ui.document.file)
24 table.insert(info, "✅ Document infos:")
25 table.insert(info, "- Dirname: " .. fileInfo.dirname)
26 table.insert(info, "- Filename: " .. fileInfo.filename)
27 table.insert(info, "- Extension: " .. fileInfo.extension)
28
29 self:info(table.concat(info, "\n"))
30endThere’s probably a simpler approach to get the book data, but nothing that a regex and some willpower can’t solve.
Accessing the highlights
Now that we have access to the most basic information about the current book, we need to figure out where KoReader saves the information we want from the book.
After a few minutes of research and testing, I ended up seeing that in the same folder where the book is saved, another folder called filename.sdr is generated, and in that folder, a file metadata.{book extension}.lua is created that contains all the basic information about the book.
Analyzing the metadata file, I found the annotation key that contains all the saved highlights and their information.
1function ObsidianSync:getSDRData()
2 if not self.ui.document then
3 self:info("❌ Open a document to proced")
4 return
5 end
6
7 local fileInfo = self:getFileNameAndExtension()
8 local chunk, err =
9 loadfile(fileInfo.dirname .. "/" .. fileInfo.filename .. ".sdr/metadata." .. fileInfo.extension .. ".lua")
10 if not chunk then
11 self:info("❌ Error to open sdr: " .. err)
12 return
13 end
14
15 local metadata = chunk()
16 self:info(metadata["annotations"][1]["text"])
17endDesigning server communication
Now that we have the data we need, we just need to be able to communicate with the server and send the files.
I thought of N ways we could make this connection, but I decided to focus on the basics first and then improve it if necessary. So, to start, we just need the IP of the machine where the server will be running and the PORT where the service will be listening.
Before trying to communicate with the server, I decided to create a basic configuration file so the user can fill in this information once and reuse it in future reading sessions.
1local ObsidianSync = WidgetContainer:extend({
2 name = "obsidiansync",
3 is_doc_only = false,
4
5 defaults = {
6 address = "127.0.0.1",
7 port = 9090,
8 password = "",
9 },
10})
11
12function ObsidianSync:configure(touchmenu_instance)
13 local MultiInputDialog = require("ui/widget/multiinputdialog")
14 local url_dialog
15
16 local current_settings = self.settings
17 or G_reader_settings:readSetting("obsidiansync_settings", ObsidianSync.defaults)
18
19 local obsidian_url_address = current_settings.address
20 local obsidian_url_port = current_settings.port
21
22 url_dialog = MultiInputDialog:new({
23 title = _("Set custom obsidian address"),
24 fields = {
25 {
26 text = obsidian_url_address,
27 input_type = "string",
28 hint = _("IP Address"),
29 },
30 {
31 text = tostring(obsidian_url_port),
32 input_type = "number",
33 hint = _("Port"),
34 },
35 },
36 buttons = {
37 {
38 {
39 text = _("Cancel"),
40 id = "close",
41 callback = function()
42 UIManager:close(url_dialog)
43 end,
44 },
45 {
46 text = _("OK"),
47 callback = function()
48 local fields = url_dialog:getFields()
49 if fields[1] ~= "" then
50 local port = tonumber(fields[2])
51 if not port or port < 1 or port > 65355 then
52 port = ObsidianSync.defaults.port
53 end
54
55 -- Preserves existing device_id when saving
56 local new_settings = {
57 address = fields[1],
58 port = port,
59 device_id = self.settings.device_id,
60 }
61 G_reader_settings:saveSetting("obsidiansync_settings", new_settings)
62 self.settings = new_settings
63 self:showNotification(_("✅ Settings saved!"))
64 end
65 UIManager:close(url_dialog)
66 if touchmenu_instance then
67 touchmenu_instance:updateItems()
68 end
69 end,
70 },
71 },
72 },
73 })
74 UIManager:show(url_dialog)
75 url_dialog:onShowKeyboard()
76endThis method is responsible for creating the inputs and saving the data filled in by the user in a configuration file that persists independently of the current book or session.
Now that we have the data for where to send the files, we can just grab the metadata and send it to the server — the metadata is basically an object (table) and we’ll let the server handle which data it wants or not.
1function ObsidianSync:sendToServer()
2 local json = require("json")
3
4 local metadata = self:getSDRData()
5 if not metadata then
6 return
7 end
8
9 local body, err = json.encode(metadata)
10 if not body then
11 self:showNotification(_("❌ Error encoding JSON: ") .. (err or _("unknown")), 5)
12 return
13 end
14
15 local device_id = self:getDeviceID()
16 if not device_id or device_id == "" then
17 self:showNotification(_("❌ Error: Could not get device ID."), 5)
18 return
19 end
20
21 local settings = self.settings or G_reader_settings:readSetting("obsidiansync_settings", ObsidianSync.defaults)
22 local url = "http://" .. settings.address .. ":" .. settings.port .. "/sync"
23
24 self:showNotification(_("Syncing with server..."), 2)
25
26 UIManager:scheduleIn(0.25, function()
27 self:_doSyncRequest(url, body, device_id)
28 end)
29endHere we grab all the metadata and, with the user’s saved settings, communicate with the server at the /sync route to complete the highlights sync.
Conclusion
We’ve reached the end of the plugin development (at least the KoReader part) and the complete code is available on my github in the koreadersync.koplugin repository.