Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .scalafmt.conf
Original file line number Diff line number Diff line change
@@ -1,2 +1,5 @@
version = "3.11.5"
runner.dialect = scala3
runner.dialect = scala3
maxColumn = 120
rewrite.scala3.endMarker.insert.blankGaps.min = 1
rewrite.scala3.endMarker.remove.blankGaps.max = 0
53 changes: 23 additions & 30 deletions cli/src/main/scala/marklit/cli/Daemon.scala
Original file line number Diff line number Diff line change
Expand Up @@ -8,28 +8,24 @@ import java.nio.file.Paths

/** Long-lived stdin/stdout JSON-RPC server.
*
* Started by `marklit --daemon`. Reads one request per line from stdin,
* dispatches to a handler, writes one response per line to stdout. The loop
* ends on any of:
* Started by `marklit --daemon`. Reads one request per line from stdin, dispatches to a handler, writes one response
* per line to stdout. The loop ends on any of:
*
* - stdin closes (parent process death) — primary shutdown trigger; the
* `Console.readLine` effect surfaces it via `IOException`/EOF.
* - idle timeout expires — defense in depth against hung-but-not-dead
* parents.
* - stdin closes (parent process death) — primary shutdown trigger; the `Console.readLine` effect surfaces it via
* `IOException`/EOF.
* - idle timeout expires — defense in depth against hung-but-not-dead parents.
* - explicit `shutdown` RPC.
*/
object Daemon:

/** 15 minutes — long enough for sbt to come back from a `~test` cycle, short
* enough that an orphaned daemon doesn't sit forever.
/** 15 minutes — long enough for sbt to come back from a `~test` cycle, short enough that an orphaned daemon doesn't
* sit forever.
*/
val defaultIdleTimeout: Duration = 15.minutes

/** The original stdout FileDescriptor, captured before we redirect
* `System.out` to stderr. The JSON-RPC reply path writes through this
* reference so user-pipeline `Console.printLine`/`println` calls — which now
* go through the redirected `System.out` — can't corrupt the wire format no
* matter what classpath or library does the printing.
/** The original stdout FileDescriptor, captured before we redirect `System.out` to stderr. The JSON-RPC reply path
* writes through this reference so user-pipeline `Console.printLine`/`println` calls — which now go through the
* redirected `System.out` — can't corrupt the wire format no matter what classpath or library does the printing.
*/
private lazy val protocolOut: java.io.PrintStream =
new java.io.PrintStream(
Expand Down Expand Up @@ -74,20 +70,17 @@ object Daemon:
// which is correct: by here, the daemon's main fiber has already
// resolved cleanly and we just need the JVM to follow.
.ensuring(ZIO.succeed(java.lang.Runtime.getRuntime.halt(0)))
end run

/** Race [[serveOnce]] against an idle timer. Whichever fiber lands first
* publishes its outcome to a Promise; the loser is fire-and-forget
* interrupted.
/** Race [[serveOnce]] against an idle timer. Whichever fiber lands first publishes its outcome to a Promise; the
* loser is fire-and-forget interrupted.
*
* Why a Promise instead of `serveOnce.race(timer)`: the reader fiber's
* `Console.readLine` ultimately blocks in `System.in.read()` on a non-daemon
* JVM thread that the kernel can't be interrupted out of. `race` (and
* `raceFirst`, and `join.race(join)`) all want to either await the loser's
* interruption or observe both completions in lockstep — empirically that
* hangs when the loser is wedged in a syscall, even if the winner has
* already produced a result. Going through `await` (which yields an `Exit`
* and never throws) and feeding the first landing into a Promise
* short-circuits that.
* Why a Promise instead of `serveOnce.race(timer)`: the reader fiber's `Console.readLine` ultimately blocks in
* `System.in.read()` on a non-daemon JVM thread that the kernel can't be interrupted out of. `race` (and
* `raceFirst`, and `join.race(join)`) all want to either await the loser's interruption or observe both completions
* in lockstep — empirically that hangs when the loser is wedged in a syscall, even if the winner has already
* produced a result. Going through `await` (which yields an `Exit` and never throws) and feeding the first landing
* into a Promise short-circuits that.
*/
private def serveOneRequest(
idleTimeout: Duration
Expand Down Expand Up @@ -115,10 +108,9 @@ object Daemon:
private enum LoopOutcome:
case Continue, Stop

/** Read one request, handle it, write one response. Returns
* [[LoopOutcome.Continue]] for normal requests, [[LoopOutcome.Stop]] for
* `shutdown`. EOF on stdin (parent closed our pipe) surfaces as an
* `EOFException` that the outer `run` catches.
/** Read one request, handle it, write one response. Returns [[LoopOutcome.Continue]] for normal requests,
* [[LoopOutcome.Stop]] for `shutdown`. EOF on stdin (parent closed our pipe) surfaces as an `EOFException` that the
* outer `run` catches.
*/
private def serveOnce: ZIO[Any, Throwable, LoopOutcome] =
Console.readLine.flatMap { line =>
Expand Down Expand Up @@ -225,3 +217,4 @@ object Daemon:
protocolOut.println(resp.toJson)
protocolOut.flush()
}
end Daemon
32 changes: 15 additions & 17 deletions cli/src/main/scala/marklit/cli/DaemonProtocol.scala
Original file line number Diff line number Diff line change
Expand Up @@ -4,23 +4,19 @@ import zio.json.*

/** JSON-RPC protocol for marklit's daemon mode.
*
* One request per line over stdin, one response per line over stdout. The
* structure is intentionally close to [[MarklitOptions]] so a daemon request
* is roughly equivalent to "the CLI options for one invocation." Fields that
* only make sense at the CLI level (e.g. `daemon`, `idle-timeout`, `watch`)
* are not part of the protocol.
* One request per line over stdin, one response per line over stdout. The structure is intentionally close to
* [[MarklitOptions]] so a daemon request is roughly equivalent to "the CLI options for one invocation." Fields that
* only make sense at the CLI level (e.g. `daemon`, `idle-timeout`, `watch`) are not part of the protocol.
*
* Methods:
* - `compile-document` — process one or more markdown files; returns a
* [[CompileResponse]] with per-file success.
* - `clear-cache` — placeholder for the disk cache landing in Phase B.
* Currently a no-op that returns `ok`.
* - `compile-document` — process one or more markdown files; returns a [[CompileResponse]] with per-file success.
* - `clear-cache` — placeholder for the disk cache landing in Phase B. Currently a no-op that returns `ok`.
* - `shutdown` — daemon writes an ack and exits its loop.
*/
object DaemonProtocol:

/** Tag-discriminated request union. zio-json's `@jsonDiscriminator` lets us
* use a sum-of-records ADT without writing AST-level codecs.
/** Tag-discriminated request union. zio-json's `@jsonDiscriminator` lets us use a sum-of-records ADT without writing
* AST-level codecs.
*/
@jsonDiscriminator("method")
sealed trait Request
Expand All @@ -33,16 +29,17 @@ object DaemonProtocol:
case object Shutdown extends Request

given JsonCodec[Request] = DeriveJsonCodec.gen[Request]
end Request

/** Parameters for the `clear-cache` RPC. The daemon doesn't keep cache state
* between requests; the caller tells it which directory to clear.
/** Parameters for the `clear-cache` RPC. The daemon doesn't keep cache state between requests; the caller tells it
* which directory to clear.
*/
final case class ClearCacheParams(cacheDir: String)
object ClearCacheParams:
given JsonCodec[ClearCacheParams] = DeriveJsonCodec.gen[ClearCacheParams]

/** A single compile request. Mirrors the subset of [[MarklitOptions]] that
* makes sense over the wire. Paths are strings (the daemon resolves them).
/** A single compile request. Mirrors the subset of [[MarklitOptions]] that makes sense over the wire. Paths are
* strings (the daemon resolves them).
*/
final case class CompileParams(
inputFiles: List[String],
Expand All @@ -68,12 +65,12 @@ object DaemonProtocol:
sealed trait Response
object Response:
@jsonHint("ok")
final case class Ok(method: String, result: Option[CompileResponse] = None)
extends Response
final case class Ok(method: String, result: Option[CompileResponse] = None) extends Response
@jsonHint("error")
final case class Err(method: String, message: String) extends Response

given JsonCodec[Response] = DeriveJsonCodec.gen[Response]
end Response

/** Per-file outcome from one compile-document request. */
final case class FileResult(
Expand All @@ -89,3 +86,4 @@ object DaemonProtocol:
def isSuccess: Boolean = files.forall(_.success)
object CompileResponse:
given JsonCodec[CompileResponse] = DeriveJsonCodec.gen[CompileResponse]
end DaemonProtocol
6 changes: 4 additions & 2 deletions cli/src/main/scala/marklit/cli/MarklitCli.scala
Original file line number Diff line number Diff line change
Expand Up @@ -251,8 +251,7 @@ object MarklitCli extends ZIOCliDefault:
effect.as(options)
}

/** Split a `--classpath`-style flag value into entries (colon/semicolon
* separated), or empty when absent.
/** Split a `--classpath`-style flag value into entries (colon/semicolon separated), or empty when absent.
*/
private def splitClasspath(cp: Option[String]): Vector[String] =
cp.map(_.split("[;:]").toVector).getOrElse(Vector.empty)
Expand Down Expand Up @@ -307,3 +306,6 @@ object MarklitCli extends ZIOCliDefault:
.fail(new RuntimeException(s"${result.failedCount} file(s) failed"))
.unless(result.success)
yield ()
end for
end runMarklit
end MarklitCli
Original file line number Diff line number Diff line change
Expand Up @@ -167,3 +167,4 @@ object DependencyResolverSpec extends ZIOSpecDefault:
}
)
)
end DependencyResolverSpec
Original file line number Diff line number Diff line change
@@ -1,25 +1,18 @@
package marklit.compiler.shim

import marklit.compiler.api.{
CompileRequest,
CompileResponse,
Diag,
DotcInvoker,
Severity
}
import marklit.compiler.api.{CompileRequest, CompileResponse, Diag, DotcInvoker, Severity}

import scala.collection.mutable.ArrayBuffer
import scala.jdk.CollectionConverters._
import scala.tools.nsc.reporters.StoreReporter
import scala.tools.nsc.{Global, Settings}

/** The 2.13 sibling of `DotcInvokerImpl`. The ONLY class in marklit that
* imports `scala.tools.nsc.*`. Compiled into a small jar bundled as a CLI
* resource and loaded at runtime from a per-version URLClassLoader holding the
/** The 2.13 sibling of `DotcInvokerImpl`. The ONLY class in marklit that imports `scala.tools.nsc.*`. Compiled into a
* small jar bundled as a CLI resource and loaded at runtime from a per-version URLClassLoader holding the
* user-requested `scala-compiler:2.13.x`.
*
* Reuses the version-neutral `DotcInvoker` interface — the name is dotc-
* historical but the surface fits both compilers.
* Reuses the version-neutral `DotcInvoker` interface — the name is dotc- historical but the surface fits both
* compilers.
*/
final class NscInvokerImpl extends DotcInvoker {

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,24 +3,16 @@ package marklit.compiler.shim
import dotty.tools.dotc.Driver
import dotty.tools.dotc.core.Contexts.Context
import dotty.tools.dotc.reporting.{Diagnostic, Reporter}
import marklit.compiler.api.{
CompileRequest,
CompileResponse,
Diag,
DotcInvoker,
Severity
}
import marklit.compiler.api.{CompileRequest, CompileResponse, Diag, DotcInvoker, Severity}

import scala.collection.mutable.ArrayBuffer
import scala.jdk.CollectionConverters.*

/** The shim implementation. This is the ONLY class in marklit that imports
* `dotty.tools.dotc.*`. It is compiled into a small jar that travels with the
* CLI as a resource, and at runtime it is loaded from a custom URLClassLoader
/** The shim implementation. This is the ONLY class in marklit that imports `dotty.tools.dotc.*`. It is compiled into a
* small jar that travels with the CLI as a resource, and at runtime it is loaded from a custom URLClassLoader
* containing the user-requested `scala3-compiler` version.
*
* The class must be public and have a no-arg constructor so the orchestrator
* can instantiate it via
* The class must be public and have a no-arg constructor so the orchestrator can instantiate it via
* `Class.forName(...).getDeclaredConstructor().newInstance()`.
*/
final class DotcInvokerImpl extends DotcInvoker:
Expand Down Expand Up @@ -56,12 +48,12 @@ final class DotcInvokerImpl extends DotcInvoker:
.asJava

new CompileResponse(!reporter.hasErrors, diags)
end compile

end DotcInvokerImpl

/** Diagnostic record local to the shim. Kept Scala-native for ergonomic
* collection here; we convert to the version-neutral `Diag` POJO at the
* boundary in `compile`.
/** Diagnostic record local to the shim. Kept Scala-native for ergonomic collection here; we convert to the
* version-neutral `Diag` POJO at the boundary in `compile`.
*/
private final case class ShimDiag(
severity: Severity,
Expand All @@ -73,9 +65,8 @@ private final case class ShimDiag(

/** Reporter that collects every diagnostic dotc emits during compilation.
*
* Lives in the shim because it extends a dotc class — the JVM resolves
* `Reporter` (and `Diagnostic`, `Context`) through this class's defining
* classloader, which contains the per-version `scala3-compiler` jar.
* Lives in the shim because it extends a dotc class — the JVM resolves `Reporter` (and `Diagnostic`, `Context`)
* through this class's defining classloader, which contains the per-version `scala3-compiler` jar.
*/
private final class CollectingReporter extends Reporter:
private val _diagnostics = ArrayBuffer.empty[ShimDiag]
Expand All @@ -102,3 +93,5 @@ private final class CollectingReporter extends Reporter:
else None

_diagnostics += ShimDiag(severity, dia.message, line, column, file)
end doReport
end CollectingReporter
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,17 @@ import zio.test.*
import java.nio.file.{Files, Path}
import scala.jdk.CollectionConverters.*

/** Integration tests for the shim. The shim is the only place dotty.* is
* touched, so these tests are the only place we verify a real compilation
* against a real dotc — every other module mocks at the `Compiler` trait.
/** Integration tests for the shim. The shim is the only place dotty.* is touched, so these tests are the only place we
* verify a real compilation against a real dotc — every other module mocks at the `Compiler` trait.
*
* The shim is exercised here within sbt's test classloader, which has the
* project's compile-time `scala3-compiler` on it. Cross-version loading
* happens later, in CompilerFactory's tests.
* The shim is exercised here within sbt's test classloader, which has the project's compile-time `scala3-compiler` on
* it. Cross-version loading happens later, in CompilerFactory's tests.
*/
object DotcInvokerImplSpec extends ZIOSpecDefault:

/** Resolve scala3-library + scala-library jars via Coursier, against the
* version the shim was compiled with. This is the same approach the
* production code (CompilerFactory) takes — keeping the test honest about
* the same boundary the real classpath construction uses.
/** Resolve scala3-library + scala-library jars via Coursier, against the version the shim was compiled with. This is
* the same approach the production code (CompilerFactory) takes — keeping the test honest about the same boundary
* the real classpath construction uses.
*/
private lazy val scalaLibraryJars: Vector[String] =
val version = new DotcInvokerImpl().compilerVersion()
Expand All @@ -35,9 +32,8 @@ object DotcInvokerImplSpec extends ZIOSpecDefault:
.toVector
.map(_.getAbsolutePath)

/** Allocate a writable temp dir and a `.scala` source file with the given
* content, then build a CompileRequest pointing at them. Returns the request
* plus the temp dir so the caller can clean up.
/** Allocate a writable temp dir and a `.scala` source file with the given content, then build a CompileRequest
* pointing at them. Returns the request plus the temp dir so the caller can clean up.
*/
private def mkRequest(source: String): (CompileRequest, Path) =
val tmp = Files.createTempDirectory("marklit-shim-test-")
Expand Down Expand Up @@ -140,3 +136,4 @@ object DotcInvokerImplSpec extends ZIOSpecDefault:
finally cleanup(tmp)
}
)
end DotcInvokerImplSpec
Loading
Loading