Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,7 @@ dist/
#Added by cargo

/target
.DS_Store
.DS_Store

# Local Kindle books may contain copyrighted text and signed delivery metadata.
/fixtures/moto-jurnory/
52 changes: 50 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,21 +19,58 @@ cat "My Clippings.txt" | ck-cli parse > output.json

# Extract unique titles
cat "My Clippings.txt" | ck-cli parse | jq -r .[].title | sort -u

# Extract highlights from a mounted Kindle
ck-cli sdr --path "/Volumes/Kindle/documents"

# Extract one book as ClippingItem JSON
ck-cli sdr --path "/path/to/Book.sdr" --json
```

**Options:**
**Parse options:**

- `-i, --input`: Input file path (default: stdin)
- `-o, --output`: Output file path or `http` for web sync (default: stdout)

**Output format:**

```json
[{
"title": "Book Title",
"content": "Highlighted text",
"pageAt": "78",
"pageAt": "#78",
"createdAt": "2019-03-27T19:57:26Z"
}]
```

### Kindle `.sdr` Highlights

Recent Kindle sidecars store annotations as positions rather than embedding the
highlighted words. The `sdr` command pairs `.azw3r` and `.yjr` sidecars with
their sibling AZW3/KF8 or KFX books and reconstructs the selected text locally:

```bash
# Recursively scan a Kindle root or documents directory
ck-cli sdr --path "/Volumes/Kindle/documents"

# Process a single sidecar directory or book
ck-cli sdr --path "/path/to/Book.sdr"
ck-cli sdr --path "/path/to/Book.azw3" --json
ck-cli sdr --path "/path/to/Book.kfx" --json
```

`--path` accepts a Kindle root/documents tree, one `.sdr` directory, or one
`.azw3`, `.azw`, KF8-containing `.mobi`, or `.kfx` file. The default output is
readable text grouped by book. `--json` emits the same `title`, `content`,
`pageAt`, and `createdAt` schema as `parse`; printed APNX or KFX navigation pages
are preferred, with the raw annotation position used as a fallback.

The implementation is read-only, offline, and written natively in Go—Python and
KindleUnpack are not runtime dependencies. It supports unencrypted AZW3/KF8
books with `.azw3r` sidecars and unencrypted KFX books with `.yjr` sidecars.
DRM-protected books and Mobi7 remain unsupported; `.yjf` reading statistics are
ignored because they do not contain highlight selections.

### Web Sync

```bash
Expand Down Expand Up @@ -66,11 +103,22 @@ See [Makefile](./Makefile) for all commands.
- Flexible I/O (files, stdin/stdout, web sync)
- High-performance processing of large files
- Direct ClippingKK web service integration
- Native Kindle `.sdr` highlight extraction for `.azw3r` and `.yjr`
- Cross-platform (macOS, Linux, Windows)

## Contributing

See [CLAUDE.md](./CLAUDE.md) for development guidelines.

## License

[MIT](https://choosealicense.com/licenses/mit/)

The `.sdr` implementation was informed by the published format research in
[kindle-reading-dashboard](https://github.com/zevisvei/kindle-reading-dashboard)
and the container behavior documented by
[KindleUnpack](https://github.com/kevinhendricks/KindleUnpack). No code from
either GPLv3 project is bundled or required.

Binary Amazon Ion values in KFX containers are decoded with the Apache-2.0
licensed [Amazon Ion Go](https://github.com/amazon-ion/ion-go) library.
5 changes: 3 additions & 2 deletions cmd/ck-cli/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ import (
var (
// Version is set at build time
Version = "dev"
// Commit is set at build time
// Commit is set at build time
Commit = "unknown"
)

Expand Down Expand Up @@ -60,6 +60,7 @@ func main() {
Commands: []*cli.Command{
commands.LoginCommand,
commands.ParseCommand,
commands.SDRCommand,
},
Before: func(c *cli.Context) error {
// Inject global configuration context
Expand All @@ -71,4 +72,4 @@ func main() {
if err := app.RunContext(ctx, os.Args); err != nil {
log.Fatal(err)
}
}
}
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ module github.com/clippingkk/cli
go 1.24

require (
github.com/amazon-ion/ion-go v1.5.0
github.com/pelletier/go-toml/v2 v2.2.4
github.com/urfave/cli/v2 v2.27.7
)
Expand Down
18 changes: 18 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
@@ -1,10 +1,28 @@
github.com/amazon-ion/ion-go v1.5.0 h1:fxsAyFda8N9HsM2xYbQSxJ3Qi/oLn0xzLoiXWG3bseg=
github.com/amazon-ion/ion-go v1.5.0/go.mod h1:3ZEje8i20TiIPVZlN+KE3B2ppZ1B8d9F/KaT7Dtec+k=
github.com/cpuguy83/go-md2man/v2 v2.0.7 h1:zbFlGlXEAKlwXpmvle3d8Oe3YnkKIK4xSRTd3sHPnBo=
github.com/cpuguy83/go-md2man/v2 v2.0.7/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/google/go-cmp v0.5.0 h1:/QaMHBdZ26BB3SSst0Iwl10Epc+xhTquomWX0oZEB6w=
github.com/google/go-cmp v0.5.0/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/russross/blackfriday/v2 v2.1.0 h1:JIOH55/0cWyOuilr9/qlrm0BSXldqnqwMsf35Ld67mk=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/urfave/cli/v2 v2.27.7 h1:bH59vdhbjLv3LAvIu6gd0usJHgoTTPhCFib8qqOwXYU=
github.com/urfave/cli/v2 v2.27.7/go.mod h1:CyNAG/xg+iAOg0N4MPGZqVmv2rCoP267496AOXUZjA4=
github.com/xrash/smetrics v0.0.0-20250705151800-55b8f293f342 h1:FnBeRrxr7OU4VvAzt5X7s6266i6cSVkkFPS0TuXWbIg=
github.com/xrash/smetrics v0.0.0-20250705151800-55b8f293f342/go.mod h1:Ohn+xnUBiLI6FVj/9LpzZWtj1/D6lUovWYBkxHVV3aM=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
109 changes: 109 additions & 0 deletions internal/commands/sdr.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
package commands

import (
"fmt"
"io"
"os"
"strings"
"unicode/utf8"

"github.com/clippingkk/cli/internal/models"
"github.com/clippingkk/cli/internal/sdr"
"github.com/urfave/cli/v2"
)

// SDRCommand extracts highlights from Kindle .sdr sidecars and their books.
var SDRCommand = &cli.Command{
Name: "sdr",
Usage: "Extract highlighted text from Kindle .sdr sidecars",
Description: `Read Kindle .sdr sidecars and recover highlighted text from their
unencrypted AZW3/KF8 or KFX books. The path may be a mounted Kindle or documents
tree, a single .sdr directory, or a supported book.

Examples:
ck-cli sdr --path /Volumes/Kindle/documents
ck-cli sdr --path "Book.sdr" --json
ck-cli sdr --path "Book.azw3" --json
ck-cli sdr --path "Book.kfx" --json`,
Flags: []cli.Flag{
&cli.StringFlag{
Name: "path",
Aliases: []string{"p"},
Usage: "Kindle documents tree, .sdr directory, or AZW3/KF8/KFX book",
Required: true,
},
&cli.BoolFlag{
Name: "json",
Usage: "Output the existing ClippingItem JSON format",
},
},
Action: sdrAction,
}

func sdrAction(c *cli.Context) error {
report, err := sdr.ExtractPath(c.String("path"))
for _, warning := range report.Warnings {
fmt.Fprintf(os.Stderr, "⚠️ %s\n", warning)
}
if err != nil {
return err
}

count := 0
for _, book := range report.Books {
count += len(book.Highlights)
}
if c.Bool("json") {
items := make([]models.ClippingItem, 0, count)
for _, book := range report.Books {
for _, highlight := range book.Highlights {
items = append(items, models.ClippingItem{
Title: highlight.Title, Content: highlight.Text,
PageAt: highlight.PageAt, CreatedAt: highlight.CreatedAt,
})
}
}
if err := outputJSON(os.Stdout, items); err != nil {
return err
}
} else {
if err := renderSDRText(os.Stdout, report); err != nil {
return err
}
}

fmt.Fprintf(os.Stderr, "📚 Extracted %d text annotations from %d books\n", count, report.Decoded)
return nil
}

func renderSDRText(writer io.Writer, report sdr.Report) error {
for _, book := range report.Books {
if len(book.Highlights) == 0 {
continue
}
if _, err := fmt.Fprintln(writer, book.Title); err != nil {
return fmt.Errorf("write output: %w", err)
}
if _, err := fmt.Fprintln(writer, strings.Repeat("=", utf8.RuneCountInString(book.Title))); err != nil {
return fmt.Errorf("write output: %w", err)
}
if _, err := fmt.Fprintln(writer); err != nil {
return fmt.Errorf("write output: %w", err)
}
for _, highlight := range book.Highlights {
if _, err := fmt.Fprintf(writer, "[%s] %s\n", highlight.Type, highlight.Text); err != nil {
return fmt.Errorf("write output: %w", err)
}
if highlight.Note != "" {
if _, err := fmt.Fprintf(writer, " Note: %s\n", highlight.Note); err != nil {
return fmt.Errorf("write output: %w", err)
}
}
if _, err := fmt.Fprintf(writer, " Location: %s | Created: %s\n\n",
highlight.PageAt, highlight.CreatedAt.UTC().Format("2006-01-02T15:04:05Z07:00")); err != nil {
return fmt.Errorf("write output: %w", err)
}
}
}
return nil
}
27 changes: 27 additions & 0 deletions internal/commands/sdr_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
package commands

import (
"strings"
"testing"
"time"

"github.com/clippingkk/cli/internal/sdr"
)

func TestRenderSDRText(t *testing.T) {
report := sdr.Report{Books: []sdr.BookResult{{
Title: "原则",
Highlights: []sdr.Highlight{{
Type: sdr.AnnotationNote, Text: "selected text", Note: "remember",
PageAt: "#12", CreatedAt: time.Date(2026, 7, 1, 8, 30, 0, 0, time.UTC),
}},
}}}
var output strings.Builder
if err := renderSDRText(&output, report); err != nil {
t.Fatalf("renderSDRText() error = %v", err)
}
want := "原则\n==\n\n[note] selected text\n Note: remember\n Location: #12 | Created: 2026-07-01T08:30:00Z\n\n"
if output.String() != want {
t.Fatalf("renderSDRText() = %q, want %q", output.String(), want)
}
}
Loading
Loading