Skip to content

Enhance export features, refactor capabilities, and improve auth pages - #975

Open
qianmoQ wants to merge 201 commits into
devlive-community:devfrom
qianmoQ:2026.0.0
Open

qianmoQ wants to merge 201 commits into
devlive-community:devfrom
qianmoQ:2026.0.0

Conversation

@qianmoQ

@qianmoQ qianmoQ commented Nov 1, 2026

Copy link
Copy Markdown
Member

Changelog category (leave one)

  • New Feature
  • Bug Fix
  • Documentation (changelog entry is not required)
  • Other

Changelog entry (Details of this change)

If there is an issue connection, write it to the end of the question
e.g: issue-7
Please delete this information when submitting

  • e.g: Support xxxx

Affected version

  • e.g: latest version

The tooltip around a database ER card's header kept the header as wide as the table name; it
now spans the card, a long name ends in an ellipsis and the tooltip shows the full name with
the table comment.
Each plugin declares its capabilities in src/main/resources/org/devlive/metaforge/plugin/
capabilities.json instead of <ServiceClass>.capabilities.json. As every plugin uses the same
path, the SPI reads the file from the jar or classes directory of the plugin's service class
rather than by resource name, so plugins on one class path never read each other's; a
subclass whose jar has no file still uses its parent's. The tests load classes from
throwaway directories, and the plugin guide shows the path.
ConvertService gains options() and extension(): a plugin lists the settings its writer takes,
with bilingual labels and choices, and reads the values from ConvertRequest.options.
- JSON: array or JSON Lines, indentation, date format (ISO 8601, yyyy-MM-dd HH:mm:ss or epoch
  milliseconds) and whether NULL fields are kept
- CSV: delimiter, header row and NULL text; cells holding the delimiter, quotes or line breaks
  are now quoted
- TXT: delimiter, header row and NULL text
- XML: root and row element names and whether NULL attributes are kept
Export now follows the prototype in the workbench: choose objects, set
content per table, pick a format plugin with its declared settings and a
live sample, set the output and execution, then check the plan and run
it as a job whose file is downloaded when it is done. The page fits the
viewport and only the step content scrolls.
The export API lists installed formats with their declared settings,
writes a sample of a table with the chosen format, checks a plan per
table, and runs it as a job that reads tables in batches, writes one
file per table plus schema.sql, and packs them into a ZIP to download.
Nothing read the flag since plugins declare their capabilities, so the
SPI default, the four overrides and the source list field go away.
Sign-in, sign-up, setup, home, 404 and 403 had no route title, so the
browser tab only showed MetaForge. They now show their page name, and
the title is rewritten when the language changes.
Only blank strings counted as empty, so a custom configures array sent
as null reached getMapValue and testing the connection failed with an
NPE. Any field without a value is now left out, and a null array gives
no environment.
Entry pages were never kept as a return address, so a network drop on
sign-in, sign-up or setup sent the user to home after reconnecting and
sign-up was lost. The network page now keeps them with their own
redirect; signing in and the other interim pages still never return to
an entry page.
Downloads only logged a lost connection, so exporting a file while the
server was unreachable left the page in place. They now go through the
same failure handling as other requests.
System settings (stored under the SYSTEM configure category) switch on
registration, the captcha, remembered sign-ins, email sign-in, email
verification on sign-up and password reset, and hold the SMTP server
and the terms and privacy links. Email features only take effect once
mail can be sent. The sign-in page reads what is on from
/api/auth/options; codes are mailed by /api/auth/code and passwords
reset by /api/auth/reset. Users get an email column, and administrators
a System settings menu on fresh and existing installs.
Administrators turn sign-in and sign-up features on and off, set the
agreement links and the SMTP server, and send a test mail with the
server as it is in the form. Features that send mail are marked until a
mail server is set.
The auth pages share a brand panel and a card with a language menu and
sign-in / sign-up tabs. Each optional part appears only when system
settings turn it on: the captcha, remembered sign-in, sign-in by email,
the forgot password link and its reset page, email verification with a
code button that counts down, and the agreement box. Sign-up shows that
it is closed when registration is off.
…lled

MfInput passes autocomplete (still off by default) and name to its
input, so forms can mark usernames, passwords and one-time codes.
Browser autofill no longer paints its own background on the bare input
inside the field.
The auth background used min-h-screen, which style.css redefines for the
app layout, so the page stopped about 360px short of the bottom. The
username rule now sits under the field, the captcha row fills the
width, and the captcha comes after the passwords on sign-up with each
field carrying its autofill hint, so browsers stop filling the username
into it.
The released class names load the drivers again:
io.edurt.datacap.driver.redis.RedisDriver,
io.edurt.datacap.driver.cluster.RedisClusterDriver,
io.edurt.datacap.driver.MongoJdbcDriver and com.dbschema.MongoJdbcDriver.
The Redis driver takes jdbc:redis-cluster://host:port,host:port again,
reads the database from the URL path, and runs plain Redis commands as
before, answering with one RESULTS column; execute() now returns that
result. SELECT n is refused because connections are pooled.
replace the badge-only page with a complete guide: project introduction,
feature list, requirements, development and production build
instructions that match the actual build (IDE launch, dist tarball),
docker deployment and a project structure map; keep the connector
gallery, community sections and badges
Each provider is stored under the SYSTEM configure category and is offered
on the sign-in page only once an administrator turns it on and sets its
client; OpenID Connect also needs its endpoints. Client secrets are never
sent back and are kept when saved empty, and the fixed GitHub and Google
endpoints can not be changed.

/api/auth/oauth/{provider}/authorize starts the authorization code flow
with a state kept in the browser and PKCE. The callback finds the user by
the linked account (metaforge_user_oauth), then by a verified email, or
creates one while registration is open, and hands the token to the web
page through a one-time ticket. An optional site address fixes the
callback URL behind a proxy.
The development loader read target/dependency, but plugin builds copy
their jars to target/dependencies, so a plugin loaded from its module
saw no driver jar at all. Drivers on the server's own class path still
worked; the MetaForge drivers (Redis, MongoDB, Kafka, ZooKeeper, HDFS,
Aliyun OSS) failed with ClassNotFoundException. Both folders are read
now.
A class in a parent-first package that the parent loader does not have,
such as org.devlive.metaforge.RedisDriver inside a plugin's driver jar,
failed with ClassNotFoundException. The plugin's own jars are tried
before giving up.
System settings get a third-party sign-in section for each provider's
client and endpoints. The sign-in page shows the prototype's other
sign-in buttons only for providers an administrator turned on, and a
callback page signs in with the returned ticket and goes back to where
the user was.
A label around each provider field also caught clicks on the client
secret's show button; the fields are plain blocks with accessible names
now, and the read-only callback address gets one too.
Sign-in and sign-up, the mail server and third-party sign-in are tabs on
the left, each with how much of it is on; the chosen group fills the
right with its own heading and the save button. The tab is kept in the
address, and a save that fails a check opens the tab with the error.
The driver read values with Redisson's binary codec, so keys set by
redis-cli or applications failed to decode. Values are plain strings
now. SELECT takes quoted keys (`test:list`), reads an exact key, a
pattern or, as before, the keys starting with the name, keeps only the
selected columns, and returns its columns even when nothing matches.
SELECT version() reports the server's redis_version.
SpotBugs failed the test module on unclosed result sets and instanceof
assertions in the Redis compatibility tests.
RedisE2ETest writes keys with plain commands, reads them back with SQL
and browses them as the source tree does. The connector page lists the
verified versions and the SQL the driver understands.
…he source

execute() always answered false, so MetaForge, which runs every
statement through it, got an update count of 0 instead of rows. It now
leaves the result set, and an empty result still has metadata. The
database comes from the property or the URL path (admin otherwise), and
getMetaData() lists databases, collections and the fields of their
first documents through the SQL driver foundation.
The plugin passes the configured database in the URL, which it built
and then dropped, lists metadata through the driver's JDBC metadata as
the other MetaForge drivers do, and declares database:list, table:list
and column:read, so the query page and the source workbench show a
tree.
The list uses the standard page layout and toolbar instead of its own
sidebar, statistics and banner. Run details sit beside a side menu that
leads to the orchestration and the run records, like account settings.
Database and table fields list what the chosen data source holds, and
changing a field clears the ones that depend on it. Column mapping is a
table of target columns choosing incoming columns from the upstream
query, matched by name automatically. SQL fields use the query console
editor with completion and a preview of the first rows.

Validation results move to a console tab, empty inspector tabs and the
unused expanded view are gone, and the side panels share the dashboard
studio header, widths and toggles.
ClickHouse and similar engines write on executeBatch and reject commit(),
so batch writes into them failed after the rows were already stored. The
writer now commits only when the driver supports transactions, and falls
back to auto-commit if a driver rejects commit() anyway.
Executors list the metrics they report in workflow.yaml. Run details and
the editor console render exactly those, grouped into volume, throughput
and pipeline cards with a rate trend, instead of a fixed set that showed
values no executor fills. A test keeps the declarations, the web metric
registry and the translations in step.

The run details header now stays in place while the content scrolls.
Runs smaller than one batch reported zero committed batches, because only flushes during addRow were counted.
Releases and Docker start with spring.config.location pointing at the configure directory, which replaces the classpath defaults, so the bundled copy was never read. It stays in the sources for running from the IDE.
Run details show overview, metrics, log, output and history tabs under a fixed header. Metrics are drawn as a progress ring, read-versus-written bars, a rate trend with average and peak, and the reader/writer pipeline. Workflow settings separate basics from executor job settings.
Dates formatted with the browser locale showed 2026/11/5 in Chinese. A shared formatter now writes dashes and a 24-hour clock everywhere, and a test rejects locale date formatting in the sources.
History lists start and end times, and stage names, states and the timing chart use translated labels.
The column settings panel drew its own grid table; it now uses MfTable with the same row drag handling as the table structure page.
Without a pre-count the total stayed unknown even after a successful run. A finished run now reports the rows read as the exact total, marked as COMPLETED.
Charts read metrics through helpers that fall back for missing or malformed values, so other executors and older run records cannot break the run pages. The source total explains that it is known when the run ends.
Workflows pre-count the source query unless they turn it off, so the total and progress are known from the start. The settings show declared defaults, and an uncounted total reads as not counted.
Workflow run pages now use RunShell, RunMetrics, RunStages and RunLog from views/components/run so other executor-driven runs can show the same records.
Datasets synced by an executor get a sync records page with the same
layout as workflow runs: overview with state changes, the metrics and
stages the executor declares, the log and the sync history, under a
fixed header with sync now and stop. It replaces the history and log
dialogs and reuses the existing history, metrics, log and stop APIs.
The menu showed the raw state key of the dataset's step list; it now shows the latest step with the same label and color as the detail page.
MfBadge laid its count out beside the wrapped content, so the notification bell showed a tall pill next to the icon. A badge wrapping content is now placed on its top-right corner, with an optional inset; standalone badges stay inline.
Dataset notifications linked to the edit wizard. Sync notices now open the dataset's sync records, and other dataset notices open its detail page.
Add typed filters, shared result and chart components, and guarded query/report state.
Preserve legacy report settings and refresh dataset reports through the dataset API.
Remember each panel width and keep floating panels open during resizing.
Resize component categories with the library and show complete category labels.
Ad hoc queries required COMPLETE_SUCCESS, a state nothing ever set, so
the button on the detail page stayed disabled. They now need only the
dataset table, as do syncs, and rebuilding is offered only when the
build itself failed.

Each finished sync now records the data state: success completes the
dataset, a failure or timeout marks the data failed and keeps the reason
as the dataset message. A stop requested by the user changes nothing.
Ad hoc queries become Visualize, the visualization studio becomes Data Screens and visual reports become Data Reports, with the related titles and hints.
A user can make one of their published data screens their home page from
the data screen list or its preview. Home then shows that screen under
the usual navigation, with ways back to the default home, while
?default=1 still opens the default home page.

The choice is stored per user in a new home_page column. Only an owned,
published screen can be chosen, and deleting the screen clears it.
The home page no longer adds a title bar with the screen name, edit, fullscreen and unset buttons; those stay on the data screen list and preview.
Add live resource counts, bounded recent resources and navigation shortcuts.
Preserve custom home selection and handle empty, failed and stale responses.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant