Repository navigation
[Data Importer] Apply a mapping to an element without saving it - #703
Merged
markus-moser merged 32 commits intoOct 7, 2026
Merged
Conversation
Callers that want the result of a mapping without running an import, e.g. to preview it or to record the changes elsewhere, had to use internal classes. MappingApplier prepares a mapping array once and applies it to an element in memory, with no queue, save, events or application log. Reference operators can consult a caller-provided lookup, and operators that write elements are listed by lint() and refused. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Updating a data object makes the generic data index look up its siblings, and the CI job with the lowest dependencies has no reachable search index for it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Splits long lines and repeated literals flagged by the quality gate. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Callers that check a file before importing it had no way to get its rows without queueing them. SourceFileReader returns the rows an import of a CSV or XLSX file would queue. An opt-in typed mode returns stored XLSX values, so numbers, booleans and text can be told apart; imports keep receiving text. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sonar flagged the assignment in the fopen condition and the repeated test literal. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sonar allows 120 characters per line. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Advanced many-to-many relations keep only the id of a related element and load it again, so an unsaved element from a reference lookup ended in a null-pointer error. The data target now names the element and why it has to be saved, and the docs no longer suggest negative ids for them. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Load Data Object switches the global hideUnpublished flag and only set it back after the loop, so an exception from a reference lookup left it flipped for the rest of the process. It is now restored in finally. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Applying a mapping logs nothing, so an unresolved reference silently became null and could clear an existing relation without a trace. apply() now returns what the operators would have logged as MappingIssue objects; imports keep logging as before. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The example loaded the element with force, which puts the changed instance into the runtime cache the section warns about. It now applies the mapping to a copy that other code in the process does not get. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Loading by path, id and attribute and reporting a miss each get their own method, so the loader stays readable now that it also restores the unpublished flag. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Row-reading error and encoding contracts are incomplete, and several new files violate required file-header conventions.
Review effort: Balanced
Findings: 3
Open (7)
What changed in this PR
Adds an in-memory mapping API and source-row reader while preserving existing import behavior.
Changes:
- Adds mapping preparation/application, warning capture, and reference lookup APIs (
src/Mapping/Apply/*). - Adds CSV/XLSX row reading with validation and typed XLSX values.
- Separates asset loading/importing and supports unsaved many-to-many references.
Assessment:
- The implementation is at the correct mapping/interpreter boundaries and covers existing warning-producing operators (
LoadAsset.php:83-88,LoadDataObject.php:203-212,ObjectField.php:40-48). - The internal
LoadAssetinheritance change is documented (doc/01_Installation/01_Upgrade.md:22-23). - Regression coverage is broad, but unreadable-file handling and malformed UTF-8 header keys remain incomplete (
XlsxFileInterpreter.php:66,CsvFileInterpreter.php:63-65,AbstractInterpreter.php:216). - New-file requirements are also missing from
AbstractAssetOperator.phpand three test files.
| File | Description |
|---|---|
tests/unit/SourceFileReaderTest.php |
Tests CSV/XLSX row reading and failures. |
tests/unit/MappingApplierTest.php |
Tests in-memory mapping behavior. |
tests/unit/ManyToManyRelationMergeTest.php |
Tests unsaved relation merging. |
tests/unit/AttributeReferenceLookupTest.php |
Tests attribute reference queries. |
src/Resources/config/services/mapping.yml |
Registers public mapping services. |
src/Mapping/WritesElementsInterface.php |
Marks element-writing components. |
src/Mapping/Operator/Simple/ObjectField.php |
Redirects warnings during application. |
src/Mapping/Operator/Simple/LoadDataObject.php |
Adds reference lookup and state restoration. |
src/Mapping/Operator/Simple/LoadAsset.php |
Adds non-writing asset lookup. |
src/Mapping/Operator/Simple/ImportAsset.php |
Marks asset imports as writing. |
src/Mapping/Operator/Simple/AbstractAssetOperator.php |
Shares asset operator behavior. |
src/Mapping/DataTarget/ManyToManyRelation.php |
Handles unsaved relation identities. |
src/Mapping/Apply/SourceFileReader.php |
Exposes source-row reading. |
src/Mapping/Apply/ReferenceType.php |
Defines reference element types. |
src/Mapping/Apply/ReferenceQuery.php |
Defines reference lookup requests. |
src/Mapping/Apply/ReferenceLookupInterface.php |
Defines custom reference resolution. |
src/Mapping/Apply/ReferenceLoadStrategy.php |
Defines supported lookup strategies. |
src/Mapping/Apply/PreparedMapping.php |
Applies prepared mappings. |
src/Mapping/Apply/MappingIssue.php |
Represents mapping warnings/issues. |
src/Mapping/Apply/MappingApplier.php |
Builds and validates mappings. |
src/Mapping/Apply/MappingApplicationScopeAwareTrait.php |
Connects operators to application scope. |
src/Mapping/Apply/MappingApplicationScope.php |
Tracks lookup and warnings. |
src/Exception/MappingApplicationException.php |
Wraps item-level failures. |
src/DataSource/Interpreter/XlsxFileInterpreter.php |
Adds XLSX row reading. |
src/DataSource/Interpreter/RowReaderInterface.php |
Defines row-reader contract. |
src/DataSource/Interpreter/CsvFileInterpreter.php |
Adds CSV row reading. |
src/DataSource/Interpreter/AbstractInterpreter.php |
Adds shared validation helpers. |
doc/06_Extending/README.md |
Links the new extension guide. |
doc/06_Extending/03_Apply_Mapping_Without_Saving.md |
Documents the new APIs. |
doc/01_Installation/01_Upgrade.md |
Records migration and behavior changes. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An operator or data target that writes elements without WritesElementsInterface would otherwise persist data while the mapping is only applied. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The MIME type guesser threw its own exception instead of InvalidInputException. Also covers the header name check on the import path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A custom file format that overrides assertValidRowEncoding() would fail to load with the added parameter. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… code Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Applies a stored Data Importer mapping to an element in memory, without saving it, and reads the rows of a source file without importing them.
MappingApplier::lint()/prepare()build a mapping from amappingConfigarray and reject items that cannot be built or that write elements (WritesElementsInterface; Import Asset implements it).PreparedMapping::apply($element, $row, ?ReferenceLookupInterface)returns the warnings the operators would have logged aslist<MappingIssue>and wraps a failing item inMappingApplicationException, with the warnings up to it. Saving or deleting an element whileapply()runs throws.ReferenceLookupInterfacelets the caller resolve Load Data Object / Load Asset references first (ReferenceQuery: type,ReferenceLoadStrategyid/path/attribute, key, attribute settings), e.g. to unsaved elements. Merge mode of the Many-to-Many Relation data target tells elements without an id apart by instance.SourceFileReader::readRows($interpreterConfig, $path, $typedValues)returns the rows an import would queue, for CSV and XLSX (RowReaderInterface). Invalid files, a missing sheet, CSV rows that do not match the header row and rows that are not UTF-8 throwInvalidInputException, naming the row.Changes for imports (only the first needs action on upgrade, so only it is in the upgrade notes):
LoadAssetno longer extendsImportAsset; both extendAbstractAssetOperator.DataObject::getHideUnpublished()when loading throws.InvalidInputException, naming the column position, instead of failing later when the row is encoded.InvalidInputExceptioninstead of an\Errorwhen a related element cannot be loaded by id.ImportAssetinheritWritesElementsInterface. A custom operator or data target without it that writes elements fails while a mapping is applied, instead of saving.Docs:
doc/06_Extending/03_Apply_Mapping_Without_Saving.md.🤖 Generated with Claude Code
Part of pimcore/product-management#1563