diff --git a/account_invoice_import_invoice2data/README.rst b/account_invoice_import_invoice2data/README.rst new file mode 100644 index 0000000000..d1de026da8 --- /dev/null +++ b/account_invoice_import_invoice2data/README.rst @@ -0,0 +1,297 @@ +=================================== +Account Invoice Import Invoice2data +=================================== + +.. + !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! + !! This file is generated by oca-gen-addon-readme !! + !! changes will be overwritten. !! + !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! + !! source digest: sha256:c7d1d2a38227a9bf81bc6132091b5285a4afe0b1280bea7541ebab212f57f16a + !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! + +.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png + :target: https://odoo-community.org/page/development-status + :alt: Beta +.. |badge2| image:: https://img.shields.io/badge/licence-AGPL--3-blue.png + :target: http://www.gnu.org/licenses/agpl-3.0-standalone.html + :alt: License: AGPL-3 +.. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fedi-lightgray.png?logo=github + :target: https://github.com/OCA/edi/tree/16.0/account_invoice_import_invoice2data + :alt: OCA/edi +.. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png + :target: https://translation.odoo-community.org/projects/edi-16-0/edi-16-0-account_invoice_import_invoice2data + :alt: Translate me on Weblate +.. |badge5| image:: https://img.shields.io/badge/runboat-Try%20me-875A7B.png + :target: https://runboat.odoo-community.org/builds?repo=OCA/edi&target_branch=16.0 + :alt: Try me on Runboat + +|badge1| |badge2| |badge3| |badge4| |badge5| + +This module is an extension of the module *account_invoice_import*: it adds support for regular PDF invoices i.e. PDF invoice that don't have an embedded XML file. It uses the `invoice2data library `_ which takes care of extracting the text of the PDF invoice, find an existing invoice template and execute the invoice template to extract the useful information from the invoice. + +To know the full story behind the development of this module, read this `blog post `_. + +More information for creating the templates can be found in `tutorial of the invoice2data library `_. The templates have to be created manually. An graphical template creator for odoo is a work in progress. + +**WARNING**: an alternative module **account_invoice_import_simple_pdf** developped in July 2021 provides similar features but has one big advantage: the accountant can add support for a new vendor by himself, no more invoice templates which require technical skill. The module *account_invoice_import_simple_pdf* provides basic functionality, but does not support line level accounting. + +**Table of contents** + +.. contents:: + :local: + +Installation +============ + +This module requires the Python library *invoice2data* available on `Github `_ with a version >= 0.2.74 (February 2018). + +To install the latest version of this library, run: + +.. code:: + + sudo pip3 install --upgrade invoice2data + +If you use Ubuntu 16.04 LTS or higher, you can use the pdftotext version 0.41.0 that is packaged in the distribution: + +.. code:: + + sudo apt install poppler-utils + +If you want the invoice2data library to support mixed-type pdf's or fallback on OCR if the PDF doesn't contain text (only a very small minority of PDF invoices are image-based and require OCR) like scanned receipts, you should also install `Ocrmypdf `_ + +.. code:: + + pip install -U ocrmypdf + +If you want the invoice2data library to fallback on OCR if the PDF doesn't contain text (only a very small minority of PDF invoices are image-based and require OCR) like scanned receipts, you should also install `Imagemagick `_ (to get the *convert* utility to convert PDF to TIFF) and `Tesseract OCR `_ : + +.. code:: + + sudo apt install imagemagick tesseract-ocr + +If you want to use custom invoice templates for the invoice2data lib (in addition to the templates provided by the invoice2data lib), you should add a line in your Odoo server configuration file such as: + +.. code:: + + invoice2data_templates_dir = /opt/invoice2data_local_templates + +and store your invoice templates in YAML format (*.yml* extension) or json format in the directory that you have configured above. If you add invoice tempates in this directory, you don't have to restart Odoo, they will be used automatically on the next invoice import. + +If you want to use only your custom invoice templates and ignore the templates provided by the invoice2data lib, you should have in your Odoo server configuration file: + +.. code:: + + invoice2data_templates_dir = /opt/invoice2data_local_templates + invoice2data_exclude_built_in_templates = True + +The yaml templates are loaded with [pyyaml](https://github.com/yaml/pyyaml) which is a pure python implementation. (thus rather slow) +As an alternative json templates can be used. Which are natively better supported by python. The performance with yaml templates can be greatly increased **10x** by using [libyaml](https://github.com/yaml/libyaml) +It can be installed on most distributions by: + +.. code:: + + sudo apt-get libyaml-dev + +French users should also install the module *l10n_fr_business_document_import* available in the `French localization `_. +Dutch users should also install the module *l10n_nl_business_document_import* available in the `Netherlands localization `_. + +Dependencies +============ +| Name | Requirement | Installation | Description | +| -------------- | :---------: | :---------: | :-------------------------------------- | +| [invoice2data](https://github.com/invoice-x/invoice2data) | required | `pip install invoice2data` | The main dependency of this invoice import module | +| [poppler-utils](https://poppler.freedesktop.org/) | required | `apt install poppler-utils` | The default Input-reader for the invoice2data library | +| [dateparser](https://dateparser.readthedocs.io/en/latest/#) | required | `pip install dateparser` | Requirement for parsing the invoice dates, this requirement is likely already satisfied by odoo itself | +| [libyaml](https://github.com/yaml/libyaml) | optional | `apt install libyaml-dev` | Template loader, recomended to greatly speedup the loading of yaml templates | +| [imagemagick](https://www.imagemagick.org) | optional | `apt install imagemagick` | inputreader: Pre-processes the pdf before feeding it into tesseract-ocr | +| [tesseract-ocr](https://github.com/tesseract-ocr/tesseract) | optional | `apt install tesseract-ocr` | inputreader: for ocr of image only pdf files | +| [tesseract-ocr- lang](https://tesseract-ocr.github.io/tessdoc/Data-Files-in-different-versions.html) | optional | `apt install tesseract-ocr-` see documentation | inputreader: Language pack for tesseract ocr, greatly improves character detection | +| [ocrmypdf](https://github.com/ocrmypdf/OCRmyPDF) | optional | `apt install ocrmypdf` | inputreader: For image only or mixed type pdf's. It uses tesseract-ocr under the hood, but provides optimalisations which greatly improves results | + +Configuration +============= + +Go to the form view of the supplier and configure it with the following parameters: + +* the *VAT* is set (the VAT number is used by default when searching the supplier in the Odoo partner database) +* in the *Invoicing* tab, create an *Invoice Import Configuration*. + +For the PDF invoice of your supplier that don't have an embedded XML file, you will have to create a `template file `_ in YAML format in the invoice2data Python library. It is quite easy to do ; if you are familiar with `regexp `_, it should not take more than 10 minutes for each supplier. + +Here are some hints to help you add a template for your supplier: + +* There is a `tutorial in the repo of the invoice2data library `_ + +* Take `Free SAS template file `_ as an example. You will find a sample PDF invoice for this supplier under invoice2data/test/pdf/invoice_free_fiber_201507.pdf + +* Try to run the invoice2data library manually on the sample invoice of Free: + +.. code:: + + % python -m invoice2data.main --debug invoice2data/test/pdf/invoice_free_fiber_201507.pdf + +On the output, you will get first the text of the PDF, then some debug info on the parsing of the invoice and the regexps, and, on the last line, you will have the dict that contain the result of the parsing. + +* if the VAT number of the supplier is present in the text of the PDF invoice, I think it's a good idea to use it as the keyword. It is good practice to add 2 other keywords: one for the language (for example, match on the word *Invoice* in the language of the invoice) and one for the currency, to match only the invoices of that supplier in this particular language and currency. + +* the list of *fields* should contain the following entries: + + * 'vat' with the VAT number of the supplier (if the VAT number of the supplier is not in the text of PDF file, add a 'partner_name' key) + * 'amount' ('amount' is the total amount with taxes) + * 'amount_untaxed' or 'amount_tax' (one or the other, no need for both) + * 'date': the date of the invoice + * 'invoice_number' + * 'date_due', if this information is available in the text of the PDF file. + +The invoice2data library is quite powerfull. It supports multiple input methods (pdftotext, ocrmypdf, tesseract ocr, google cloud vision). +Even invoicelines can be imported and mapped to products in the database. +The invoice2data library does not have a strict standard on field names. This makes the module very flexible, but also hard to create re-usable templates. + +If you want to make use of the advanced features, support for the following fields is implemented. + +## Supported fields + +(note: the fieldname column contains the name to be used in the template file.) + +Partner fields +| fieldname | type | Description | +| -------------- | :---------: | :-------------------------------------- | +| vat | char | The vat code is unique for each partner, it has the highest priority for matching the partner | +| partner_name | char | self explaining | +| partner_street | char | self explaining | +| partner_street2 | char | self explaining | +| partner_street3 | char | self explaining | +| partner_city | char | self explaining | +| partner_zip | char | self explaining | +| country_code | char | use iso format fr or nl | +| state_code | char | use iso format NY (for New York) | +| partner_email | char | self explaining | +| partner_website | char | self explaining | +| telephone | char | can be used for matching the partner with the help of support modules | +| mobile | char | can be used for matching the partner contact with the help of support modules | +| partner_ref | char | reference name or number can be used for partner matching | +| siren | char | French business code, can be used for matching the partner | +| partner_coc | char | General business identiefier number, can be used for matching the partner | + +Invoice Fields (on document level) +| fieldname | type | Description | +| -------------- | :---------: | :-------------------------------------- | +| currency | char | The currency of the invoice in iso format (EUR, USD) | +| currency_symbol | char | The currency symbol of the invoice (€, $) | +| bic | char | Bank Identifier Code | +| iban | char | International Bank Account Number | +| amount | float | The total amount of the invoice (including taxes) | +| amount_untaxed | float | The total amount of the invoice (excluding taxes) | +| amount_tax | float | The sum of the tax amount of the invoice | +| date | date | The date of the invoice | +| invoice_number | char | self explaining | +| date_due | date | The duedate of the invoice | +| date_start | date | The start date of the period for the invoice when the services are delivered. | +| date_end | date | The start date of the period for the invoice when the services are delivered. | +| note | char | The contents of this field will be imported in the chatter. | +| narration | char | The contents of this field will be imported in the narration field. (on the bottom of the invoice.) | +| payment_reference | char | If the invoice is pre-paid an reference can be used for payment reconciliation | +| payment_unece_code | char | The unece code of the payment means according to 4461 code list | +| incoterm | char | The Incoterm 2000 abbrevation | +| company_vat | char | The vat number of the company to which the invoice is addressed to. Used to check if the invoice is actually is adressed to the company which wants to process it. (Very useful in multi-company setup) | +| mandate_id | char | A banking mandate is attached to a bank account and represents an authorization that the bank account owner gives to a company for a specific operation (such as direct debit). | + + +Invoice line Fields +| fieldname | type | Description | +| -------------- | :---------: | :-------------------------------------- | +| name | char | The name of the product, can be used for product matching | +| barcode | char | The the barcode of the product or product package, used for product matching | +| code | char | The (internal) product code, used for product matching | +| qty | float | The amount of items/units | +| unece_code | char | The unece code of the products units of measure can be passed | +| uom | char | The name of the unit of measure, internally if will be mapped to the unece code. Example L will be mapped to unece_code LTR | +| price_unit | float | The unit price of the item. (excluding taxes) | +| discount | float | The amount of discount for this line. Eg 20 for 20% discount or 0.0 for no discount | +| price_total | float | The total amount of the invoice line including taxes. It can be used to select the correct tax tag. | +| price_subtotal | float | The total amount of the invoice line excluding taxes. It can be used to create adjustment lines when the decimal precision is insufficient. | +| line_tax_percent | float | The percentage of tax | +| line_tax_amount | float | The fixed amount of tax applied to the line | +| line_note | char | Notes on the invoice can be imported, There is a special view available. | +| sectionheader | char | There is a special view available for section headers. | +| date_start | date | The start date of the period for the invoice when the services are delivered. | +| date_end | date | The start date of the period for the invoice when the services are delivered. | + +Known issues / Roadmap +====================== + +* Implement support for lines with all tax included, used for some localizations like Switzerland or scanned receipts. +* An graphical template builder. +* Once invoice2data 1.0 is published on PyPI: opt into its new APIs — the input-backend cascade (faster pdfium-first default + automatic fallback), runtime ``ai_fallback=True`` for LLM extraction when no template matches, ``raise_on_error=True`` for typed ``NoTemplateFoundError`` / ``RequiredFieldsMissingError`` instead of ``{}``, and ``pre_process_pdf`` returning the cleaned/smaller PDF so the wizard can re-attach it to the Odoo invoice in place of the raw upload. + +Known Issues +* The input module is hard coded to use pdftotext parser and as a fallback to tesseract. +* Creation of the templates is still quite hard. +* The addres and company specific fields are parsed. Meaning it is possible to import an invoice which is issued to another company than yours! + +Changelog +========= + +16.0.1.0.0 (2025-10-02) +~~~~~~~~~~~~~~~~~~~~~~~ + +* Migration to 16.0. + +14.0.2.2.0 (2023-03-03) +~~~~~~~~~~~~~~~~~~~~~~~ + +* [ADD] Support for invoicelines. + (`#74 `_) + +Bug Tracker +=========== + +Bugs are tracked on `GitHub Issues `_. +In case of trouble, please check there if your issue has already been reported. +If you spotted it first, help us to smash it by providing a detailed and welcomed +`feedback `_. + +Do not contact contributors directly about support or help with technical issues. + +Credits +======= + +Authors +~~~~~~~ + +* Akretion + +Contributors +~~~~~~~~~~~~ + +* Alexis de Lattre +* Marwan Behillil + +Maintainers +~~~~~~~~~~~ + +This module is maintained by the OCA. + +.. image:: https://odoo-community.org/logo.png + :alt: Odoo Community Association + :target: https://odoo-community.org + +OCA, or the Odoo Community Association, is a nonprofit organization whose +mission is to support the collaborative development of Odoo features and +promote its widespread use. + +.. |maintainer-alexis-via| image:: https://github.com/alexis-via.png?size=40px + :target: https://github.com/alexis-via + :alt: alexis-via +.. |maintainer-bosd| image:: https://github.com/bosd.png?size=40px + :target: https://github.com/bosd + :alt: bosd + +Current `maintainers `__: + +|maintainer-alexis-via| |maintainer-bosd| + +This module is part of the `OCA/edi `_ project on GitHub. + +You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute. diff --git a/account_invoice_import_invoice2data/__init__.py b/account_invoice_import_invoice2data/__init__.py new file mode 100644 index 0000000000..40272379f7 --- /dev/null +++ b/account_invoice_import_invoice2data/__init__.py @@ -0,0 +1 @@ +from . import wizard diff --git a/account_invoice_import_invoice2data/__manifest__.py b/account_invoice_import_invoice2data/__manifest__.py new file mode 100644 index 0000000000..862ba77a66 --- /dev/null +++ b/account_invoice_import_invoice2data/__manifest__.py @@ -0,0 +1,25 @@ +# Copyright 2015-2021 Akretion France (http://www.akretion.com/) +# @author: Alexis de Lattre +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). + +{ + "name": "Account Invoice Import Invoice2data", + "version": "16.0.1.0.0", + "category": "Accounting/Accounting", + "license": "AGPL-3", + "summary": "Import supplier invoices using the invoice2data lib", + "author": "Akretion,Odoo Community Association (OCA)", + "maintainers": ["alexis-via", "bosd"], + "website": "https://github.com/OCA/edi", + "depends": ["account_invoice_import"], + "external_dependencies": { + "python": [ + "invoice2data", + "dateparser", + ], + "deb": ["poppler-utils"], + }, + "data": ["wizard/account_invoice_import_view.xml"], + "demo": ["demo/demo_data.xml"], + "installable": True, +} diff --git a/account_invoice_import_invoice2data/demo/demo_data.xml b/account_invoice_import_invoice2data/demo/demo_data.xml new file mode 100644 index 0000000000..e9cd35cb93 --- /dev/null +++ b/account_invoice_import_invoice2data/demo/demo_data.xml @@ -0,0 +1,63 @@ + + + + + Internet Access + + + + service + + + + Free SAS + + 1 + 0 + 8 rue de la Ville l’Évèque + 75008 + Paris + + http://www.free.fr + FR60421938861 + + Fiber optic access at the main office + + + + Beeswax XL + Acme beeswax + 17589684 + 42 + consu + + + + + + Olive Oil + + + + consu + + + 987123 + + + + Luxury Truffles + + + + consu + + + LUX_TRF + + + + diff --git a/account_invoice_import_invoice2data/i18n/account_invoice_import_invoice2data.pot b/account_invoice_import_invoice2data/i18n/account_invoice_import_invoice2data.pot new file mode 100644 index 0000000000..7f1e5b3fcf --- /dev/null +++ b/account_invoice_import_invoice2data/i18n/account_invoice_import_invoice2data.pot @@ -0,0 +1,101 @@ +# Translation of Odoo Server. +# This file contains the translation of the following modules: +# * account_invoice_import_invoice2data +# +msgid "" +msgstr "" +"Project-Id-Version: Odoo Server 14.0\n" +"Report-Msgid-Bugs-To: \n" +"Last-Translator: \n" +"Language-Team: \n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: \n" +"Plural-Forms: \n" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "regular PDF file without any embedded XML file via the" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,description_sale:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,description_sale:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "Acme beeswax" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,name:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "Beeswax XL" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import__display_name +msgid "Display Name" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import__id +msgid "ID" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.internet_access +#: model:product.template,name:account_invoice_import_invoice2data.internet_access_product_template +msgid "Internet Access" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import____last_update +msgid "Last Modified on" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.luxury_truffles +#: model:product.template,name:account_invoice_import_invoice2data.luxury_truffles_product_template +msgid "Luxury Truffles" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.olive_oil +#: model:product.template,name:account_invoice_import_invoice2data.olive_oil_product_template +msgid "Olive Oil" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: code:addons/account_invoice_import_invoice2data/wizard/account_invoice_import.py:0 +#: code:addons/account_invoice_import_invoice2data/wizard/account_invoice_import.py:0 +#, python-format +msgid "PDF Invoice parsing failed. Error message: %s" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,uom_name:account_invoice_import_invoice2data.internet_access +#: model:product.template,uom_name:account_invoice_import_invoice2data.internet_access_product_template +msgid "Units" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model,name:account_invoice_import_invoice2data.model_account_invoice_import +msgid "Wizard to import supplier invoices/refunds" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "invoice2data" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,uom_name:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,uom_name:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "kg" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "" +"library (reads the text of the PDF and get the information). It will work " +"only if the invoice2data library has a template for the invoices of that " +"supplier." +msgstr "" diff --git a/account_invoice_import_invoice2data/i18n/de.po b/account_invoice_import_invoice2data/i18n/de.po new file mode 100644 index 0000000000..42769fbc03 --- /dev/null +++ b/account_invoice_import_invoice2data/i18n/de.po @@ -0,0 +1,115 @@ +# Translation of Odoo Server. +# This file contains the translation of the following modules: +# * account_invoice_import_invoice2data +# +msgid "" +msgstr "" +"Project-Id-Version: Odoo Server 10.0\n" +"Report-Msgid-Bugs-To: \n" +"PO-Revision-Date: 2020-05-27 09:19+0000\n" +"Last-Translator: Maria Sparenberg \n" +"Language-Team: none\n" +"Language: de\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: \n" +"Plural-Forms: nplurals=2; plural=n != 1;\n" +"X-Generator: Weblate 3.10\n" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "regular PDF file without any embedded XML file via the" +msgstr "reguläre PDF Datei ohne eingebettete XML Datei per" + +#. module: account_invoice_import_invoice2data +#: model:product.product,description_sale:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,description_sale:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "Acme beeswax" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,name:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "Beeswax XL" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import__display_name +msgid "Display Name" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import__id +msgid "ID" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.internet_access +#: model:product.template,name:account_invoice_import_invoice2data.internet_access_product_template +msgid "Internet Access" +msgstr "Internet Zugang" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import____last_update +msgid "Last Modified on" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.luxury_truffles +#: model:product.template,name:account_invoice_import_invoice2data.luxury_truffles_product_template +msgid "Luxury Truffles" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.olive_oil +#: model:product.template,name:account_invoice_import_invoice2data.olive_oil_product_template +msgid "Olive Oil" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: code:addons/account_invoice_import_invoice2data/wizard/account_invoice_import.py:0 +#, python-format +msgid "PDF Invoice parsing failed. Error message: %s" +msgstr "PDF Rechnungsverarbeitung ist gescheitert. Fehler Meldung: %s" + +#. module: account_invoice_import_invoice2data +#: model:product.product,uom_name:account_invoice_import_invoice2data.internet_access +#: model:product.template,uom_name:account_invoice_import_invoice2data.internet_access_product_template +msgid "Units" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model,name:account_invoice_import_invoice2data.model_account_invoice_import +msgid "Wizard to import supplier invoices/refunds" +msgstr "Assistent für den Import von Lieferantenrechnungen / -gutschriften" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "invoice2data" +msgstr "invoice2data" + +#. module: account_invoice_import_invoice2data +#: model:product.product,uom_name:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,uom_name:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "kg" +msgstr "kg" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "" +"library (reads the text of the PDF and get the information). It will work " +"only if the invoice2data library has a template for the invoices of that " +"supplier." +msgstr "" +"Bibliothek (liest den Text der PDF und erhält die Informationen). Dies " +"funktioniert nur, wenn die Bibliothek invoice2data eine Vorlage für die " +"Rechnungen dieses Lieferanten enthält." + +#, python-format +#~ msgid "" +#~ "This PDF invoice doesn't match a known template of the invoice2data lib." +#~ msgstr "" +#~ "Diese PDF Rechnung entspricht keinem Template der invoice2data Bibliothek." + +#~ msgid "Unit(s)" +#~ msgstr "Einheit(en)" diff --git a/account_invoice_import_invoice2data/i18n/es.po b/account_invoice_import_invoice2data/i18n/es.po new file mode 100644 index 0000000000..f69c6b55f4 --- /dev/null +++ b/account_invoice_import_invoice2data/i18n/es.po @@ -0,0 +1,118 @@ +# Translation of Odoo Server. +# This file contains the translation of the following modules: +# * account_invoice_import_invoice2data +# +# Translators: +# enjolras , 2018 +msgid "" +msgstr "" +"Project-Id-Version: Odoo Server 10.0\n" +"Report-Msgid-Bugs-To: \n" +"POT-Creation-Date: 2018-03-12 01:43+0000\n" +"PO-Revision-Date: 2023-11-23 14:34+0000\n" +"Last-Translator: Ivorra78 \n" +"Language-Team: Spanish (https://www.transifex.com/oca/teams/23907/es/)\n" +"Language: es\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: \n" +"Plural-Forms: nplurals=2; plural=n != 1;\n" +"X-Generator: Weblate 4.17\n" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "regular PDF file without any embedded XML file via the" +msgstr "" +"archivo PDF normal sin ningún archivo XML incrustado a través de la " +"función" + +#. module: account_invoice_import_invoice2data +#: model:product.product,description_sale:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,description_sale:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "Acme beeswax" +msgstr "Cera de abejas Acme" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,name:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "Beeswax XL" +msgstr "Cera de abejas XL" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import__display_name +msgid "Display Name" +msgstr "Mostrar Nombre" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import__id +msgid "ID" +msgstr "ID" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.internet_access +#: model:product.template,name:account_invoice_import_invoice2data.internet_access_product_template +msgid "Internet Access" +msgstr "Acceso a internet" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import____last_update +msgid "Last Modified on" +msgstr "Última Modificación el" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.luxury_truffles +#: model:product.template,name:account_invoice_import_invoice2data.luxury_truffles_product_template +msgid "Luxury Truffles" +msgstr "Trufas de Lujo" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.olive_oil +#: model:product.template,name:account_invoice_import_invoice2data.olive_oil_product_template +msgid "Olive Oil" +msgstr "Aceite de Oliva" + +#. module: account_invoice_import_invoice2data +#: code:addons/account_invoice_import_invoice2data/wizard/account_invoice_import.py:0 +#, python-format +msgid "PDF Invoice parsing failed. Error message: %s" +msgstr "Error en el análisis de la factura PDF. Mensaje de error: %s" + +#. module: account_invoice_import_invoice2data +#: model:product.product,uom_name:account_invoice_import_invoice2data.internet_access +#: model:product.template,uom_name:account_invoice_import_invoice2data.internet_access_product_template +msgid "Units" +msgstr "Unidades" + +#. module: account_invoice_import_invoice2data +#: model:ir.model,name:account_invoice_import_invoice2data.model_account_invoice_import +msgid "Wizard to import supplier invoices/refunds" +msgstr "Asistente para importar facturas/reembolsos de proveedores" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "invoice2data" +msgstr "invoice2data" + +#. module: account_invoice_import_invoice2data +#: model:product.product,uom_name:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,uom_name:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "kg" +msgstr "kg" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "" +"library (reads the text of the PDF and get the information). It will work " +"only if the invoice2data library has a template for the invoices of that " +"supplier." +msgstr "" +"(lee el texto del PDF y obtiene la información). Sólo funcionará si la " +"biblioteca invoice2data tiene una plantilla para las facturas de ese " +"proveedor." + +#, python-format +#~ msgid "" +#~ "This PDF invoice doesn't match a known template of the invoice2data lib." +#~ msgstr "" +#~ "Esta factura en PDF no coincide con una plantilla conocida de la " +#~ "biblioteca de invoice2data." diff --git a/account_invoice_import_invoice2data/i18n/fr.po b/account_invoice_import_invoice2data/i18n/fr.po new file mode 100644 index 0000000000..2c2ceae733 --- /dev/null +++ b/account_invoice_import_invoice2data/i18n/fr.po @@ -0,0 +1,113 @@ +# Translation of Odoo Server. +# This file contains the translation of the following modules: +# * account_invoice_import_invoice2data +# +msgid "" +msgstr "" +"Project-Id-Version: Odoo Server 14.0\n" +"Report-Msgid-Bugs-To: \n" +"PO-Revision-Date: 2023-05-29 11:08+0000\n" +"Last-Translator: Claude R Perrin \n" +"Language-Team: none\n" +"Language: fr\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: \n" +"Plural-Forms: nplurals=2; plural=n > 1;\n" +"X-Generator: Weblate 4.17\n" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "regular PDF file without any embedded XML file via the" +msgstr "fichier PDF standard sans aucun fichier XML intégré à l’aide de" + +#. module: account_invoice_import_invoice2data +#: model:product.product,description_sale:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,description_sale:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "Acme beeswax" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,name:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "Beeswax XL" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import__display_name +msgid "Display Name" +msgstr "Nom affiché" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import__id +msgid "ID" +msgstr "ID" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.internet_access +#: model:product.template,name:account_invoice_import_invoice2data.internet_access_product_template +msgid "Internet Access" +msgstr "Accès internet" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import____last_update +msgid "Last Modified on" +msgstr "Dernière modification le" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.luxury_truffles +#: model:product.template,name:account_invoice_import_invoice2data.luxury_truffles_product_template +msgid "Luxury Truffles" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.olive_oil +#: model:product.template,name:account_invoice_import_invoice2data.olive_oil_product_template +msgid "Olive Oil" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: code:addons/account_invoice_import_invoice2data/wizard/account_invoice_import.py:0 +#, python-format +msgid "PDF Invoice parsing failed. Error message: %s" +msgstr "L'analyse de la facture PDF a échoué. Message d'erreur : %s" + +#. module: account_invoice_import_invoice2data +#: model:product.product,uom_name:account_invoice_import_invoice2data.internet_access +#: model:product.template,uom_name:account_invoice_import_invoice2data.internet_access_product_template +msgid "Units" +msgstr "Unités" + +#. module: account_invoice_import_invoice2data +#: model:ir.model,name:account_invoice_import_invoice2data.model_account_invoice_import +msgid "Wizard to import supplier invoices/refunds" +msgstr "Assistant pour importer les factures/avoirs des fournisseurs" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "invoice2data" +msgstr "invoice2data" + +#. module: account_invoice_import_invoice2data +#: model:product.product,uom_name:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,uom_name:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "kg" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "" +"library (reads the text of the PDF and get the information). It will work " +"only if the invoice2data library has a template for the invoices of that " +"supplier." +msgstr "" +"bibliothèque (qui lit le texte du PDF et trouve l'information). Il ne " +"fonctionnera que si la bibliothèque invoice2data dispose d'un modèle pour " +"les factures de ce fournisseur." + +#, python-format +#~ msgid "" +#~ "This PDF invoice doesn't match a known template of the invoice2data lib." +#~ msgstr "" +#~ "Cette facture PDF ne correspond pas à un modèle connu de la bibliothèque " +#~ "invoice2data." diff --git a/account_invoice_import_invoice2data/i18n/it.po b/account_invoice_import_invoice2data/i18n/it.po new file mode 100644 index 0000000000..070d2e5f3c --- /dev/null +++ b/account_invoice_import_invoice2data/i18n/it.po @@ -0,0 +1,104 @@ +# Translation of Odoo Server. +# This file contains the translation of the following modules: +# * account_invoice_import_invoice2data +# +msgid "" +msgstr "" +"Project-Id-Version: Odoo Server 14.0\n" +"Report-Msgid-Bugs-To: \n" +"PO-Revision-Date: 2025-05-29 16:26+0000\n" +"Last-Translator: mymage \n" +"Language-Team: none\n" +"Language: it\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: \n" +"Plural-Forms: nplurals=2; plural=n != 1;\n" +"X-Generator: Weblate 5.10.4\n" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "regular PDF file without any embedded XML file via the" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,description_sale:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,description_sale:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "Acme beeswax" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,name:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "Beeswax XL" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import__display_name +msgid "Display Name" +msgstr "Nome Visualizzato" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import__id +msgid "ID" +msgstr "ID" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.internet_access +#: model:product.template,name:account_invoice_import_invoice2data.internet_access_product_template +msgid "Internet Access" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import____last_update +msgid "Last Modified on" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.luxury_truffles +#: model:product.template,name:account_invoice_import_invoice2data.luxury_truffles_product_template +msgid "Luxury Truffles" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.olive_oil +#: model:product.template,name:account_invoice_import_invoice2data.olive_oil_product_template +msgid "Olive Oil" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: code:addons/account_invoice_import_invoice2data/wizard/account_invoice_import.py:0 +#: code:addons/account_invoice_import_invoice2data/wizard/account_invoice_import.py:0 +#, python-format +msgid "PDF Invoice parsing failed. Error message: %s" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,uom_name:account_invoice_import_invoice2data.internet_access +#: model:product.template,uom_name:account_invoice_import_invoice2data.internet_access_product_template +msgid "Units" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model,name:account_invoice_import_invoice2data.model_account_invoice_import +msgid "Wizard to import supplier invoices/refunds" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "invoice2data" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,uom_name:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,uom_name:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "kg" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "" +"library (reads the text of the PDF and get the information). It will work " +"only if the invoice2data library has a template for the invoices of that " +"supplier." +msgstr "" diff --git a/account_invoice_import_invoice2data/i18n/nl.po b/account_invoice_import_invoice2data/i18n/nl.po new file mode 100644 index 0000000000..83a339fe96 --- /dev/null +++ b/account_invoice_import_invoice2data/i18n/nl.po @@ -0,0 +1,116 @@ +# Translation of Odoo Server. +# This file contains the translation of the following modules: +# * account_invoice_import_invoice2data +# +msgid "" +msgstr "" +"Project-Id-Version: Odoo Server 12.0\n" +"Report-Msgid-Bugs-To: \n" +"PO-Revision-Date: 2023-11-18 16:34+0000\n" +"Last-Translator: Bosd \n" +"Language-Team: none\n" +"Language: nl\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: \n" +"Plural-Forms: nplurals=2; plural=n != 1;\n" +"X-Generator: Weblate 4.17\n" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "regular PDF file without any embedded XML file via the" +msgstr "normaal PDF bestand zonder een geintegreerd XML bestand via de" + +#. module: account_invoice_import_invoice2data +#: model:product.product,description_sale:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,description_sale:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "Acme beeswax" +msgstr "Acme beeswax" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,name:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "Beeswax XL" +msgstr "Beeswax XL" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import__display_name +msgid "Display Name" +msgstr "Weergavenaam" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import__id +msgid "ID" +msgstr "ID" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.internet_access +#: model:product.template,name:account_invoice_import_invoice2data.internet_access_product_template +msgid "Internet Access" +msgstr "Internet Toegang" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import____last_update +msgid "Last Modified on" +msgstr "Laatst bijgewerkt op" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.luxury_truffles +#: model:product.template,name:account_invoice_import_invoice2data.luxury_truffles_product_template +msgid "Luxury Truffles" +msgstr "Luxury Truffles" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.olive_oil +#: model:product.template,name:account_invoice_import_invoice2data.olive_oil_product_template +msgid "Olive Oil" +msgstr "Olive Oil" + +#. module: account_invoice_import_invoice2data +#: code:addons/account_invoice_import_invoice2data/wizard/account_invoice_import.py:0 +#, python-format +msgid "PDF Invoice parsing failed. Error message: %s" +msgstr "Het is niet gelukt om de PDF factuur te verwerken. Fout melding: %s" + +#. module: account_invoice_import_invoice2data +#: model:product.product,uom_name:account_invoice_import_invoice2data.internet_access +#: model:product.template,uom_name:account_invoice_import_invoice2data.internet_access_product_template +msgid "Units" +msgstr "Stuks" + +#. module: account_invoice_import_invoice2data +#: model:ir.model,name:account_invoice_import_invoice2data.model_account_invoice_import +msgid "Wizard to import supplier invoices/refunds" +msgstr "Hulp voor het importeren van inkoop / credit facturen" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "invoice2data" +msgstr "invoice2data" + +#. module: account_invoice_import_invoice2data +#: model:product.product,uom_name:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,uom_name:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "kg" +msgstr "kg" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "" +"library (reads the text of the PDF and get the information). It will work " +"only if the invoice2data library has a template for the invoices of that " +"supplier." +msgstr "" +"bibliotheek (extraheert de teksten in de PDF voor het verkrijgen van de " +"informatie). Het werkt enkel wanneer er een sjabloon beschikbaar is voor de " +"betreffende leverancier." + +#, python-format +#~ msgid "" +#~ "This PDF invoice doesn't match a known template of the invoice2data lib." +#~ msgstr "" +#~ "Deze PDF factuur heeft geen overeenkomstig herkennings sjabloon van de " +#~ "invoice2data library." + +#~ msgid "Unit(s)" +#~ msgstr "Stuk(s)" diff --git a/account_invoice_import_invoice2data/i18n/pt.po b/account_invoice_import_invoice2data/i18n/pt.po new file mode 100644 index 0000000000..7f35cb520e --- /dev/null +++ b/account_invoice_import_invoice2data/i18n/pt.po @@ -0,0 +1,111 @@ +# Translation of Odoo Server. +# This file contains the translation of the following modules: +# * account_invoice_import_invoice2data +# +msgid "" +msgstr "" +"Project-Id-Version: Odoo Server 10.0\n" +"Report-Msgid-Bugs-To: \n" +"PO-Revision-Date: 2018-12-17 16:58+0000\n" +"Last-Translator: Pedro Castro Silva \n" +"Language-Team: none\n" +"Language: pt\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: \n" +"Plural-Forms: nplurals=2; plural=n > 1;\n" +"X-Generator: Weblate 3.3\n" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "regular PDF file without any embedded XML file via the" +msgstr "arquivo PDF regular sem qualquer arquivo XML incorporado via" + +#. module: account_invoice_import_invoice2data +#: model:product.product,description_sale:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,description_sale:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "Acme beeswax" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,name:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "Beeswax XL" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import__display_name +msgid "Display Name" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import__id +msgid "ID" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.internet_access +#: model:product.template,name:account_invoice_import_invoice2data.internet_access_product_template +msgid "Internet Access" +msgstr "Acesso à Internet" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import____last_update +msgid "Last Modified on" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.luxury_truffles +#: model:product.template,name:account_invoice_import_invoice2data.luxury_truffles_product_template +msgid "Luxury Truffles" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.olive_oil +#: model:product.template,name:account_invoice_import_invoice2data.olive_oil_product_template +msgid "Olive Oil" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: code:addons/account_invoice_import_invoice2data/wizard/account_invoice_import.py:0 +#, python-format +msgid "PDF Invoice parsing failed. Error message: %s" +msgstr "Análise de fatura PDF falhou. Mensagem de erro: %s" + +#. module: account_invoice_import_invoice2data +#: model:product.product,uom_name:account_invoice_import_invoice2data.internet_access +#: model:product.template,uom_name:account_invoice_import_invoice2data.internet_access_product_template +msgid "Units" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model,name:account_invoice_import_invoice2data.model_account_invoice_import +msgid "Wizard to import supplier invoices/refunds" +msgstr "Assistente para importar as faturas/créditos do fornecedor" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "invoice2data" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,uom_name:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,uom_name:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "kg" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "" +"library (reads the text of the PDF and get the information). It will work " +"only if the invoice2data library has a template for the invoices of that " +"supplier." +msgstr "" +"biblioteca (lê o texto do PDF e obtém a informação). Apenas funcionará se a " +"biblioteca de invoice2data tiver um modelo para as faturas deste fornecedor." + +#, python-format +#~ msgid "" +#~ "This PDF invoice doesn't match a known template of the invoice2data lib." +#~ msgstr "" +#~ "Esta fatura PDF não combina com um modelo conhecido do invoice2data lib." diff --git a/account_invoice_import_invoice2data/i18n/sl.po b/account_invoice_import_invoice2data/i18n/sl.po new file mode 100644 index 0000000000..f0486253e7 --- /dev/null +++ b/account_invoice_import_invoice2data/i18n/sl.po @@ -0,0 +1,113 @@ +# Translation of Odoo Server. +# This file contains the translation of the following modules: +# * account_invoice_import_invoice2data +# +# Translators: +# Matjaž Mozetič , 2016 +msgid "" +msgstr "" +"Project-Id-Version: Odoo Server 8.0\n" +"Report-Msgid-Bugs-To: \n" +"POT-Creation-Date: 2016-11-12 13:37+0000\n" +"PO-Revision-Date: 2016-11-12 13:37+0000\n" +"Last-Translator: Matjaž Mozetič , 2016\n" +"Language-Team: Slovenian (https://www.transifex.com/oca/teams/23907/sl/)\n" +"Language: sl\n" +"MIME-Version: 1.0\n" +"Content-Type: text/plain; charset=UTF-8\n" +"Content-Transfer-Encoding: \n" +"Plural-Forms: nplurals=4; plural=(n%100==1 ? 0 : n%100==2 ? 1 : n%100==3 || n" +"%100==4 ? 2 : 3);\n" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "regular PDF file without any embedded XML file via the" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,description_sale:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,description_sale:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "Acme beeswax" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,name:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "Beeswax XL" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import__display_name +msgid "Display Name" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import__id +msgid "ID" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.internet_access +#: model:product.template,name:account_invoice_import_invoice2data.internet_access_product_template +msgid "Internet Access" +msgstr "Internetni dostop" + +#. module: account_invoice_import_invoice2data +#: model:ir.model.fields,field_description:account_invoice_import_invoice2data.field_account_invoice_import____last_update +msgid "Last Modified on" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.luxury_truffles +#: model:product.template,name:account_invoice_import_invoice2data.luxury_truffles_product_template +msgid "Luxury Truffles" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,name:account_invoice_import_invoice2data.olive_oil +#: model:product.template,name:account_invoice_import_invoice2data.olive_oil_product_template +msgid "Olive Oil" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: code:addons/account_invoice_import_invoice2data/wizard/account_invoice_import.py:0 +#, python-format +msgid "PDF Invoice parsing failed. Error message: %s" +msgstr "Neuspešna obdelava PDF računa. Napaka: %s" + +#. module: account_invoice_import_invoice2data +#: model:product.product,uom_name:account_invoice_import_invoice2data.internet_access +#: model:product.template,uom_name:account_invoice_import_invoice2data.internet_access_product_template +msgid "Units" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:ir.model,name:account_invoice_import_invoice2data.model_account_invoice_import +msgid "Wizard to import supplier invoices/refunds" +msgstr "Čarovnik za uvoz prejetih računov/dobropisov" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "invoice2data" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model:product.product,uom_name:account_invoice_import_invoice2data.product_beeswax_xl +#: model:product.template,uom_name:account_invoice_import_invoice2data.product_beeswax_xl_product_template +msgid "kg" +msgstr "" + +#. module: account_invoice_import_invoice2data +#: model_terms:ir.ui.view,arch_db:account_invoice_import_invoice2data.account_invoice_import_form +msgid "" +"library (reads the text of the PDF and get the information). It will work " +"only if the invoice2data library has a template for the invoices of that " +"supplier." +msgstr "" + +#, python-format +#~ msgid "" +#~ "This PDF invoice doesn't match a known template of the invoice2data lib." +#~ msgstr "" +#~ "Ta PDF račun se ne ujema z nobeno znano predlogo iz knjižnice " +#~ "invoice2data." diff --git a/account_invoice_import_invoice2data/readme/CONFIGURE.rst b/account_invoice_import_invoice2data/readme/CONFIGURE.rst new file mode 100644 index 0000000000..c61fa8cdb7 --- /dev/null +++ b/account_invoice_import_invoice2data/readme/CONFIGURE.rst @@ -0,0 +1,105 @@ +Go to the form view of the supplier and configure it with the following parameters: + +* the *VAT* is set (the VAT number is used by default when searching the supplier in the Odoo partner database) +* in the *Invoicing* tab, create an *Invoice Import Configuration*. + +For the PDF invoice of your supplier that don't have an embedded XML file, you will have to create a `template file `_ in YAML format in the invoice2data Python library. It is quite easy to do ; if you are familiar with `regexp `_, it should not take more than 10 minutes for each supplier. + +Here are some hints to help you add a template for your supplier: + +* There is a `tutorial in the repo of the invoice2data library `_ + +* Take `Free SAS template file `_ as an example. You will find a sample PDF invoice for this supplier under invoice2data/test/pdf/invoice_free_fiber_201507.pdf + +* Try to run the invoice2data library manually on the sample invoice of Free: + +.. code:: + + % python -m invoice2data.main --debug invoice2data/test/pdf/invoice_free_fiber_201507.pdf + +On the output, you will get first the text of the PDF, then some debug info on the parsing of the invoice and the regexps, and, on the last line, you will have the dict that contain the result of the parsing. + +* if the VAT number of the supplier is present in the text of the PDF invoice, I think it's a good idea to use it as the keyword. It is good practice to add 2 other keywords: one for the language (for example, match on the word *Invoice* in the language of the invoice) and one for the currency, to match only the invoices of that supplier in this particular language and currency. + +* the list of *fields* should contain the following entries: + + * 'vat' with the VAT number of the supplier (if the VAT number of the supplier is not in the text of PDF file, add a 'partner_name' key) + * 'amount' ('amount' is the total amount with taxes) + * 'amount_untaxed' or 'amount_tax' (one or the other, no need for both) + * 'date': the date of the invoice + * 'invoice_number' + * 'date_due', if this information is available in the text of the PDF file. + +The invoice2data library is quite powerfull. It supports multiple input methods (pdftotext, ocrmypdf, tesseract ocr, google cloud vision). +Even invoicelines can be imported and mapped to products in the database. +The invoice2data library does not have a strict standard on field names. This makes the module very flexible, but also hard to create re-usable templates. + +If you want to make use of the advanced features, support for the following fields is implemented. + +## Supported fields + +(note: the fieldname column contains the name to be used in the template file.) + +Partner fields +| fieldname | type | Description | +| -------------- | :---------: | :-------------------------------------- | +| vat | char | The vat code is unique for each partner, it has the highest priority for matching the partner | +| partner_name | char | self explaining | +| partner_street | char | self explaining | +| partner_street2 | char | self explaining | +| partner_street3 | char | self explaining | +| partner_city | char | self explaining | +| partner_zip | char | self explaining | +| country_code | char | use iso format fr or nl | +| state_code | char | use iso format NY (for New York) | +| partner_email | char | self explaining | +| partner_website | char | self explaining | +| telephone | char | can be used for matching the partner with the help of support modules | +| mobile | char | can be used for matching the partner contact with the help of support modules | +| partner_ref | char | reference name or number can be used for partner matching | +| siren | char | French business code, can be used for matching the partner | +| partner_coc | char | General business identiefier number, can be used for matching the partner | + +Invoice Fields (on document level) +| fieldname | type | Description | +| -------------- | :---------: | :-------------------------------------- | +| currency | char | The currency of the invoice in iso format (EUR, USD) | +| currency_symbol | char | The currency symbol of the invoice (€, $) | +| bic | char | Bank Identifier Code | +| iban | char | International Bank Account Number | +| amount | float | The total amount of the invoice (including taxes) | +| amount_untaxed | float | The total amount of the invoice (excluding taxes) | +| amount_tax | float | The sum of the tax amount of the invoice | +| date | date | The date of the invoice | +| invoice_number | char | self explaining | +| date_due | date | The duedate of the invoice | +| date_start | date | The start date of the period for the invoice when the services are delivered. | +| date_end | date | The start date of the period for the invoice when the services are delivered. | +| note | char | The contents of this field will be imported in the chatter. | +| narration | char | The contents of this field will be imported in the narration field. (on the bottom of the invoice.) | +| payment_reference | char | If the invoice is pre-paid an reference can be used for payment reconciliation | +| payment_unece_code | char | The unece code of the payment means according to 4461 code list | +| incoterm | char | The Incoterm 2000 abbrevation | +| company_vat | char | The vat number of the company to which the invoice is addressed to. Used to check if the invoice is actually is adressed to the company which wants to process it. (Very useful in multi-company setup) | +| mandate_id | char | A banking mandate is attached to a bank account and represents an authorization that the bank account owner gives to a company for a specific operation (such as direct debit). | + + +Invoice line Fields +| fieldname | type | Description | +| -------------- | :---------: | :-------------------------------------- | +| name | char | The name of the product, can be used for product matching | +| barcode | char | The the barcode of the product or product package, used for product matching | +| code | char | The (internal) product code, used for product matching | +| qty | float | The amount of items/units | +| unece_code | char | The unece code of the products units of measure can be passed | +| uom | char | The name of the unit of measure, internally if will be mapped to the unece code. Example L will be mapped to unece_code LTR | +| price_unit | float | The unit price of the item. (excluding taxes) | +| discount | float | The amount of discount for this line. Eg 20 for 20% discount or 0.0 for no discount | +| price_total | float | The total amount of the invoice line including taxes. It can be used to select the correct tax tag. | +| price_subtotal | float | The total amount of the invoice line excluding taxes. It can be used to create adjustment lines when the decimal precision is insufficient. | +| line_tax_percent | float | The percentage of tax | +| line_tax_amount | float | The fixed amount of tax applied to the line | +| line_note | char | Notes on the invoice can be imported, There is a special view available. | +| sectionheader | char | There is a special view available for section headers. | +| date_start | date | The start date of the period for the invoice when the services are delivered. | +| date_end | date | The start date of the period for the invoice when the services are delivered. | diff --git a/account_invoice_import_invoice2data/readme/CONTRIBUTORS.rst b/account_invoice_import_invoice2data/readme/CONTRIBUTORS.rst new file mode 100644 index 0000000000..74bdeae091 --- /dev/null +++ b/account_invoice_import_invoice2data/readme/CONTRIBUTORS.rst @@ -0,0 +1,2 @@ +* Alexis de Lattre +* Marwan Behillil diff --git a/account_invoice_import_invoice2data/readme/DESCRIPTION.rst b/account_invoice_import_invoice2data/readme/DESCRIPTION.rst new file mode 100644 index 0000000000..0ab0c6f4e9 --- /dev/null +++ b/account_invoice_import_invoice2data/readme/DESCRIPTION.rst @@ -0,0 +1,7 @@ +This module is an extension of the module *account_invoice_import*: it adds support for regular PDF invoices i.e. PDF invoice that don't have an embedded XML file. It uses the `invoice2data library `_ which takes care of extracting the text of the PDF invoice, find an existing invoice template and execute the invoice template to extract the useful information from the invoice. + +To know the full story behind the development of this module, read this `blog post `_. + +More information for creating the templates can be found in `tutorial of the invoice2data library `_. The templates have to be created manually. An graphical template creator for odoo is a work in progress. + +**WARNING**: an alternative module **account_invoice_import_simple_pdf** developped in July 2021 provides similar features but has one big advantage: the accountant can add support for a new vendor by himself, no more invoice templates which require technical skill. The module *account_invoice_import_simple_pdf* provides basic functionality, but does not support line level accounting. diff --git a/account_invoice_import_invoice2data/readme/HISTORY.rst b/account_invoice_import_invoice2data/readme/HISTORY.rst new file mode 100644 index 0000000000..a8381cc2c6 --- /dev/null +++ b/account_invoice_import_invoice2data/readme/HISTORY.rst @@ -0,0 +1,10 @@ +16.0.1.0.0 (2025-10-02) +~~~~~~~~~~~~~~~~~~~~~~~ + +* Migration to 16.0. + +14.0.2.2.0 (2023-03-03) +~~~~~~~~~~~~~~~~~~~~~~~ + +* [ADD] Support for invoicelines. + (`#74 `_) diff --git a/account_invoice_import_invoice2data/readme/INSTALL.rst b/account_invoice_import_invoice2data/readme/INSTALL.rst new file mode 100644 index 0000000000..5e09952057 --- /dev/null +++ b/account_invoice_import_invoice2data/readme/INSTALL.rst @@ -0,0 +1,64 @@ +This module requires the Python library *invoice2data* available on `Github `_ with a version >= 0.2.74 (February 2018). + +To install the latest version of this library, run: + +.. code:: + + sudo pip3 install --upgrade invoice2data + +If you use Ubuntu 16.04 LTS or higher, you can use the pdftotext version 0.41.0 that is packaged in the distribution: + +.. code:: + + sudo apt install poppler-utils + +If you want the invoice2data library to support mixed-type pdf's or fallback on OCR if the PDF doesn't contain text (only a very small minority of PDF invoices are image-based and require OCR) like scanned receipts, you should also install `Ocrmypdf `_ + +.. code:: + + pip install -U ocrmypdf + +If you want the invoice2data library to fallback on OCR if the PDF doesn't contain text (only a very small minority of PDF invoices are image-based and require OCR) like scanned receipts, you should also install `Imagemagick `_ (to get the *convert* utility to convert PDF to TIFF) and `Tesseract OCR `_ : + +.. code:: + + sudo apt install imagemagick tesseract-ocr + +If you want to use custom invoice templates for the invoice2data lib (in addition to the templates provided by the invoice2data lib), you should add a line in your Odoo server configuration file such as: + +.. code:: + + invoice2data_templates_dir = /opt/invoice2data_local_templates + +and store your invoice templates in YAML format (*.yml* extension) or json format in the directory that you have configured above. If you add invoice tempates in this directory, you don't have to restart Odoo, they will be used automatically on the next invoice import. + +If you want to use only your custom invoice templates and ignore the templates provided by the invoice2data lib, you should have in your Odoo server configuration file: + +.. code:: + + invoice2data_templates_dir = /opt/invoice2data_local_templates + invoice2data_exclude_built_in_templates = True + +The yaml templates are loaded with [pyyaml](https://github.com/yaml/pyyaml) which is a pure python implementation. (thus rather slow) +As an alternative json templates can be used. Which are natively better supported by python. The performance with yaml templates can be greatly increased **10x** by using [libyaml](https://github.com/yaml/libyaml) +It can be installed on most distributions by: + +.. code:: + + sudo apt-get libyaml-dev + +French users should also install the module *l10n_fr_business_document_import* available in the `French localization `_. +Dutch users should also install the module *l10n_nl_business_document_import* available in the `Netherlands localization `_. + +Dependencies +============ +| Name | Requirement | Installation | Description | +| -------------- | :---------: | :---------: | :-------------------------------------- | +| [invoice2data](https://github.com/invoice-x/invoice2data) | required | `pip install invoice2data` | The main dependency of this invoice import module | +| [poppler-utils](https://poppler.freedesktop.org/) | required | `apt install poppler-utils` | The default Input-reader for the invoice2data library | +| [dateparser](https://dateparser.readthedocs.io/en/latest/#) | required | `pip install dateparser` | Requirement for parsing the invoice dates, this requirement is likely already satisfied by odoo itself | +| [libyaml](https://github.com/yaml/libyaml) | optional | `apt install libyaml-dev` | Template loader, recomended to greatly speedup the loading of yaml templates | +| [imagemagick](https://www.imagemagick.org) | optional | `apt install imagemagick` | inputreader: Pre-processes the pdf before feeding it into tesseract-ocr | +| [tesseract-ocr](https://github.com/tesseract-ocr/tesseract) | optional | `apt install tesseract-ocr` | inputreader: for ocr of image only pdf files | +| [tesseract-ocr- lang](https://tesseract-ocr.github.io/tessdoc/Data-Files-in-different-versions.html) | optional | `apt install tesseract-ocr-` see documentation | inputreader: Language pack for tesseract ocr, greatly improves character detection | +| [ocrmypdf](https://github.com/ocrmypdf/OCRmyPDF) | optional | `apt install ocrmypdf` | inputreader: For image only or mixed type pdf's. It uses tesseract-ocr under the hood, but provides optimalisations which greatly improves results | diff --git a/account_invoice_import_invoice2data/readme/ROADMAP.rst b/account_invoice_import_invoice2data/readme/ROADMAP.rst new file mode 100644 index 0000000000..065d95d03b --- /dev/null +++ b/account_invoice_import_invoice2data/readme/ROADMAP.rst @@ -0,0 +1,8 @@ +* Implement support for lines with all tax included, used for some localizations like Switzerland or scanned receipts. +* An graphical template builder. +* Once invoice2data 1.0 is published on PyPI: opt into its new APIs — the input-backend cascade (faster pdfium-first default + automatic fallback), runtime ``ai_fallback=True`` for LLM extraction when no template matches, ``raise_on_error=True`` for typed ``NoTemplateFoundError`` / ``RequiredFieldsMissingError`` instead of ``{}``, and ``pre_process_pdf`` returning the cleaned/smaller PDF so the wizard can re-attach it to the Odoo invoice in place of the raw upload. + +Known Issues +* The input module is hard coded to use pdftotext parser and as a fallback to tesseract. +* Creation of the templates is still quite hard. +* The addres and company specific fields are parsed. Meaning it is possible to import an invoice which is issued to another company than yours! diff --git a/account_invoice_import_invoice2data/static/description/icon.png b/account_invoice_import_invoice2data/static/description/icon.png new file mode 100644 index 0000000000..3a0328b516 Binary files /dev/null and b/account_invoice_import_invoice2data/static/description/icon.png differ diff --git a/account_invoice_import_invoice2data/static/description/index.html b/account_invoice_import_invoice2data/static/description/index.html new file mode 100644 index 0000000000..b7f4a9a411 --- /dev/null +++ b/account_invoice_import_invoice2data/static/description/index.html @@ -0,0 +1,616 @@ + + + + + +Account Invoice Import Invoice2data + + + +
+

Account Invoice Import Invoice2data

+ + +

Beta License: AGPL-3 OCA/edi Translate me on Weblate Try me on Runboat

+

This module is an extension of the module account_invoice_import: it adds support for regular PDF invoices i.e. PDF invoice that don’t have an embedded XML file. It uses the invoice2data library which takes care of extracting the text of the PDF invoice, find an existing invoice template and execute the invoice template to extract the useful information from the invoice.

+

To know the full story behind the development of this module, read this blog post.

+

More information for creating the templates can be found in tutorial of the invoice2data library. The templates have to be created manually. An graphical template creator for odoo is a work in progress.

+

WARNING: an alternative module account_invoice_import_simple_pdf developped in July 2021 provides similar features but has one big advantage: the accountant can add support for a new vendor by himself, no more invoice templates which require technical skill. The module account_invoice_import_simple_pdf provides basic functionality, but does not support line level accounting.

+

Table of contents

+ +
+

Installation

+

This module requires the Python library invoice2data available on Github with a version >= 0.2.74 (February 2018).

+

To install the latest version of this library, run:

+
+sudo pip3 install --upgrade invoice2data
+
+

If you use Ubuntu 16.04 LTS or higher, you can use the pdftotext version 0.41.0 that is packaged in the distribution:

+
+sudo apt install poppler-utils
+
+

If you want the invoice2data library to support mixed-type pdf’s or fallback on OCR if the PDF doesn’t contain text (only a very small minority of PDF invoices are image-based and require OCR) like scanned receipts, you should also install Ocrmypdf

+
+pip install -U ocrmypdf
+
+

If you want the invoice2data library to fallback on OCR if the PDF doesn’t contain text (only a very small minority of PDF invoices are image-based and require OCR) like scanned receipts, you should also install Imagemagick (to get the convert utility to convert PDF to TIFF) and Tesseract OCR :

+
+sudo apt install imagemagick tesseract-ocr
+
+

If you want to use custom invoice templates for the invoice2data lib (in addition to the templates provided by the invoice2data lib), you should add a line in your Odoo server configuration file such as:

+
+invoice2data_templates_dir = /opt/invoice2data_local_templates
+
+

and store your invoice templates in YAML format (.yml extension) or json format in the directory that you have configured above. If you add invoice tempates in this directory, you don’t have to restart Odoo, they will be used automatically on the next invoice import.

+

If you want to use only your custom invoice templates and ignore the templates provided by the invoice2data lib, you should have in your Odoo server configuration file:

+
+invoice2data_templates_dir = /opt/invoice2data_local_templates
+invoice2data_exclude_built_in_templates = True
+
+

The yaml templates are loaded with [pyyaml](https://github.com/yaml/pyyaml) which is a pure python implementation. (thus rather slow) +As an alternative json templates can be used. Which are natively better supported by python. The performance with yaml templates can be greatly increased 10x by using [libyaml](https://github.com/yaml/libyaml) +It can be installed on most distributions by:

+
+sudo apt-get libyaml-dev
+
+

French users should also install the module l10n_fr_business_document_import available in the French localization. +Dutch users should also install the module l10n_nl_business_document_import available in the Netherlands localization.

+
+
+

Dependencies

+
+
Name | Requirement | Installation | Description |
+
————– | :———: | :———: | :————————————– |
+
[invoice2data](https://github.com/invoice-x/invoice2data) | required | pip install invoice2data | The main dependency of this invoice import module |
+
[poppler-utils](https://poppler.freedesktop.org/) | required | apt install poppler-utils | The default Input-reader for the invoice2data library |
+
[dateparser](https://dateparser.readthedocs.io/en/latest/#) | required | pip install dateparser | Requirement for parsing the invoice dates, this requirement is likely already satisfied by odoo itself |
+
[libyaml](https://github.com/yaml/libyaml) | optional | apt install libyaml-dev | Template loader, recomended to greatly speedup the loading of yaml templates |
+
[imagemagick](https://www.imagemagick.org) | optional | apt install imagemagick | inputreader: Pre-processes the pdf before feeding it into tesseract-ocr |
+
[tesseract-ocr](https://github.com/tesseract-ocr/tesseract) | optional | apt install tesseract-ocr | inputreader: for ocr of image only pdf files |
+
[tesseract-ocr- lang](https://tesseract-ocr.github.io/tessdoc/Data-Files-in-different-versions.html) | optional | apt install tesseract-ocr- see documentation | inputreader: Language pack for tesseract ocr, greatly improves character detection |
+
[ocrmypdf](https://github.com/ocrmypdf/OCRmyPDF) | optional | apt install ocrmypdf | inputreader: For image only or mixed type pdf’s. It uses tesseract-ocr under the hood, but provides optimalisations which greatly improves results |
+
+
+
+

Configuration

+

Go to the form view of the supplier and configure it with the following parameters:

+
    +
  • the VAT is set (the VAT number is used by default when searching the supplier in the Odoo partner database)
  • +
  • in the Invoicing tab, create an Invoice Import Configuration.
  • +
+

For the PDF invoice of your supplier that don’t have an embedded XML file, you will have to create a template file in YAML format in the invoice2data Python library. It is quite easy to do ; if you are familiar with regexp, it should not take more than 10 minutes for each supplier.

+

Here are some hints to help you add a template for your supplier:

+ +
+% python -m invoice2data.main --debug invoice2data/test/pdf/invoice_free_fiber_201507.pdf
+
+

On the output, you will get first the text of the PDF, then some debug info on the parsing of the invoice and the regexps, and, on the last line, you will have the dict that contain the result of the parsing.

+
    +
  • if the VAT number of the supplier is present in the text of the PDF invoice, I think it’s a good idea to use it as the keyword. It is good practice to add 2 other keywords: one for the language (for example, match on the word Invoice in the language of the invoice) and one for the currency, to match only the invoices of that supplier in this particular language and currency.
  • +
  • the list of fields should contain the following entries:
      +
    • ‘vat’ with the VAT number of the supplier (if the VAT number of the supplier is not in the text of PDF file, add a ‘partner_name’ key)
    • +
    • ‘amount’ (‘amount’ is the total amount with taxes)
    • +
    • ‘amount_untaxed’ or ‘amount_tax’ (one or the other, no need for both)
    • +
    • ‘date’: the date of the invoice
    • +
    • ‘invoice_number’
    • +
    • ‘date_due’, if this information is available in the text of the PDF file.
    • +
    +
  • +
+

The invoice2data library is quite powerfull. It supports multiple input methods (pdftotext, ocrmypdf, tesseract ocr, google cloud vision). +Even invoicelines can be imported and mapped to products in the database. +The invoice2data library does not have a strict standard on field names. This makes the module very flexible, but also hard to create re-usable templates.

+

If you want to make use of the advanced features, support for the following fields is implemented.

+

## Supported fields

+

(note: the fieldname column contains the name to be used in the template file.)

+

Partner fields +| fieldname | type | Description | +| ————– | :———: | :————————————– | +| vat | char | The vat code is unique for each partner, it has the highest priority for matching the partner | +| partner_name | char | self explaining | +| partner_street | char | self explaining | +| partner_street2 | char | self explaining | +| partner_street3 | char | self explaining | +| partner_city | char | self explaining | +| partner_zip | char | self explaining | +| country_code | char | use iso format fr or nl | +| state_code | char | use iso format NY (for New York) | +| partner_email | char | self explaining | +| partner_website | char | self explaining | +| telephone | char | can be used for matching the partner with the help of support modules | +| mobile | char | can be used for matching the partner contact with the help of support modules | +| partner_ref | char | reference name or number can be used for partner matching | +| siren | char | French business code, can be used for matching the partner | +| partner_coc | char | General business identiefier number, can be used for matching the partner |

+

Invoice Fields (on document level) +| fieldname | type | Description | +| ————– | :———: | :————————————– | +| currency | char | The currency of the invoice in iso format (EUR, USD) | +| currency_symbol | char | The currency symbol of the invoice (€, $) | +| bic | char | Bank Identifier Code | +| iban | char | International Bank Account Number | +| amount | float | The total amount of the invoice (including taxes) | +| amount_untaxed | float | The total amount of the invoice (excluding taxes) | +| amount_tax | float | The sum of the tax amount of the invoice | +| date | date | The date of the invoice | +| invoice_number | char | self explaining | +| date_due | date | The duedate of the invoice | +| date_start | date | The start date of the period for the invoice when the services are delivered. | +| date_end | date | The start date of the period for the invoice when the services are delivered. | +| note | char | The contents of this field will be imported in the chatter. | +| narration | char | The contents of this field will be imported in the narration field. (on the bottom of the invoice.) | +| payment_reference | char | If the invoice is pre-paid an reference can be used for payment reconciliation | +| payment_unece_code | char | The unece code of the payment means according to 4461 code list | +| incoterm | char | The Incoterm 2000 abbrevation | +| company_vat | char | The vat number of the company to which the invoice is addressed to. Used to check if the invoice is actually is adressed to the company which wants to process it. (Very useful in multi-company setup) | +| mandate_id | char | A banking mandate is attached to a bank account and represents an authorization that the bank account owner gives to a company for a specific operation (such as direct debit). |

+

Invoice line Fields +| fieldname | type | Description | +| ————– | :———: | :————————————– | +| name | char | The name of the product, can be used for product matching | +| barcode | char | The the barcode of the product or product package, used for product matching | +| code | char | The (internal) product code, used for product matching | +| qty | float | The amount of items/units | +| unece_code | char | The unece code of the products units of measure can be passed | +| uom | char | The name of the unit of measure, internally if will be mapped to the unece code. Example L will be mapped to unece_code LTR | +| price_unit | float | The unit price of the item. (excluding taxes) | +| discount | float | The amount of discount for this line. Eg 20 for 20% discount or 0.0 for no discount | +| price_total | float | The total amount of the invoice line including taxes. It can be used to select the correct tax tag. | +| price_subtotal | float | The total amount of the invoice line excluding taxes. It can be used to create adjustment lines when the decimal precision is insufficient. | +| line_tax_percent | float | The percentage of tax | +| line_tax_amount | float | The fixed amount of tax applied to the line | +| line_note | char | Notes on the invoice can be imported, There is a special view available. | +| sectionheader | char | There is a special view available for section headers. | +| date_start | date | The start date of the period for the invoice when the services are delivered. | +| date_end | date | The start date of the period for the invoice when the services are delivered. |

+
+
+

Known issues / Roadmap

+
    +
  • Implement support for lines with all tax included, used for some localizations like Switzerland or scanned receipts.
  • +
  • An graphical template builder.
  • +
  • Once invoice2data 1.0 is published on PyPI: opt into its new APIs — the input-backend cascade (faster pdfium-first default + automatic fallback), runtime ai_fallback=True for LLM extraction when no template matches, raise_on_error=True for typed NoTemplateFoundError / RequiredFieldsMissingError instead of {}, and pre_process_pdf returning the cleaned/smaller PDF so the wizard can re-attach it to the Odoo invoice in place of the raw upload.
  • +
+

Known Issues +* The input module is hard coded to use pdftotext parser and as a fallback to tesseract. +* Creation of the templates is still quite hard. +* The addres and company specific fields are parsed. Meaning it is possible to import an invoice which is issued to another company than yours!

+
+
+

Changelog

+
+

16.0.1.0.0 (2025-10-02)

+
    +
  • Migration to 16.0.
  • +
+
+
+

14.0.2.2.0 (2023-03-03)

+
    +
  • [ADD] Support for invoicelines. +(#74)
  • +
+
+
+
+

Bug Tracker

+

Bugs are tracked on GitHub Issues. +In case of trouble, please check there if your issue has already been reported. +If you spotted it first, help us to smash it by providing a detailed and welcomed +feedback.

+

Do not contact contributors directly about support or help with technical issues.

+
+
+

Credits

+
+

Authors

+
    +
  • Akretion
  • +
+
+
+

Contributors

+ +
+
+

Maintainers

+

This module is maintained by the OCA.

+ +Odoo Community Association + +

OCA, or the Odoo Community Association, is a nonprofit organization whose +mission is to support the collaborative development of Odoo features and +promote its widespread use.

+

Current maintainers:

+

alexis-via bosd

+

This module is part of the OCA/edi project on GitHub.

+

You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute.

+
+
+
+ + diff --git a/account_invoice_import_invoice2data/tests/__init__.py b/account_invoice_import_invoice2data/tests/__init__.py new file mode 100644 index 0000000000..d0a138175d --- /dev/null +++ b/account_invoice_import_invoice2data/tests/__init__.py @@ -0,0 +1 @@ +from . import test_invoice_import diff --git a/account_invoice_import_invoice2data/tests/pdf/AzureInterior.pdf b/account_invoice_import_invoice2data/tests/pdf/AzureInterior.pdf new file mode 100644 index 0000000000..b342e2b02b Binary files /dev/null and b/account_invoice_import_invoice2data/tests/pdf/AzureInterior.pdf differ diff --git a/account_invoice_import_invoice2data/tests/pdf/invoice_free_fiber_201507.pdf b/account_invoice_import_invoice2data/tests/pdf/invoice_free_fiber_201507.pdf new file mode 100644 index 0000000000..adc46f510f Binary files /dev/null and b/account_invoice_import_invoice2data/tests/pdf/invoice_free_fiber_201507.pdf differ diff --git a/account_invoice_import_invoice2data/tests/test_invoice_import.py b/account_invoice_import_invoice2data/tests/test_invoice_import.py new file mode 100644 index 0000000000..6545c3f003 --- /dev/null +++ b/account_invoice_import_invoice2data/tests/test_invoice_import.py @@ -0,0 +1,531 @@ +# Copyright 2015-2021 Akretion France (www.akretion.com) +# @author: Alexis de Lattre +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). + +import base64 +import logging +from unittest import mock + +from odoo import Command, fields +from odoo.exceptions import UserError +from odoo.tests.common import TransactionCase +from odoo.tools import file_open, float_compare + +from odoo.addons.base.tests.common import DISABLED_MAIL_CONTEXT + + +class TestInvoiceImport(TransactionCase): + @classmethod + def setUpClass(cls): + super().setUpClass() + cls.env = cls.env(context=dict(cls.env.context, **DISABLED_MAIL_CONTEXT)) + frtax = cls.env["account.tax"].create( + { + "name": "French VAT purchase 20.0%", + "description": "FR-VAT-buy-20.0", + "amount": 20, + "amount_type": "percent", + "type_tax_use": "purchase", + } + ) + # Set this tax on Internet access product + internet_product = cls.env.ref( + "account_invoice_import_invoice2data.internet_access" + ) + internet_product.supplier_taxes_id = [Command.set([frtax.id])] + + def test_have_invoice2data_unavailable(self): + with mock.patch.dict("sys.modules", {"invoice2data": None}): + with self.assertLogs("", level="DEBUG") as cm: + logging.getLogger("").debug("Cannot import invoice2data") + self.assertEqual(cm.output, ["DEBUG:root:Cannot import invoice2data"]) + + def test_have_tesseract_unavailable(self): + with mock.patch.dict("sys.modules", {"tesseract": None}): + with self.assertLogs("", level="DEBUG") as cm: + logging.getLogger("").debug("Cannot import tesseract") + self.assertEqual(cm.output, ["DEBUG:root:Cannot import tesseract"]) + + def test_import_free_invoice(self): + filename = "invoice_free_fiber_201507.pdf" + f = file_open("account_invoice_import_invoice2data/tests/pdf/" + filename, "rb") + pdf_file = f.read() + invoice_file = self.env["ir.attachment"].create( + { + "name": filename, + "res_model": self.env["account.invoice.import"]._name, + "datas": base64.b64encode(pdf_file), + "type": "binary", + } + ) + wiz = self.env["account.invoice.import"].create( + { + "invoice_attachment_ids": invoice_file, + } + ) + f.close() + wiz.import_invoices() + # Check result of invoice creation + invoices = self.env["account.move"].search( + [ + ("state", "=", "draft"), + ("move_type", "=", "in_invoice"), + ("ref", "=", "562044387"), + ] + ) + self.assertEqual(len(invoices), 1) + inv = invoices[0] + self.assertEqual(inv.move_type, "in_invoice") + self.assertEqual(fields.Date.to_string(inv.invoice_date), "2015-07-02") + self.assertEqual( + inv.partner_id, self.env.ref("account_invoice_import_invoice2data.free") + ) + self.assertEqual(inv.journal_id.type, "purchase") + self.assertEqual(float_compare(inv.amount_total, 29.99, precision_digits=2), 0) + self.assertEqual( + float_compare(inv.amount_untaxed, 24.99, precision_digits=2), 0 + ) + self.assertEqual(len(inv.invoice_line_ids), 1) + iline = inv.invoice_line_ids[0] + self.assertEqual(iline.name, "Fiber optic access at the main office") + self.assertEqual( + iline.product_id, + self.env.ref("account_invoice_import_invoice2data.internet_access"), + ) + self.assertEqual(float_compare(iline.quantity, 1.0, precision_digits=0), 0) + self.assertEqual(float_compare(iline.price_unit, 24.99, precision_digits=2), 0) + + # Prepare data for next test i.e. invoice update + # (we re-use the invoice created by the first import !) + inv.write( + { + "invoice_date": False, + "ref": False, + } + ) + + # New import with update of an existing draft invoice + wiz2 = self.env["account.invoice.import"].create( + { + "invoice_attachment_ids": invoice_file, + } + ) + action = wiz2.import_invoices() + + self.assertEqual(action["params"]["next"]["res_model"], "account.move") + invoices = self.env["account.move"].search( + [ + ("state", "=", "draft"), + ("move_type", "=", "in_invoice"), + ("ref", "=", "562044387"), + ] + ) + self.assertEqual(len(invoices), 1) + inv = invoices[0] + self.assertEqual(fields.Date.to_string(inv.invoice_date), "2015-07-02") + + def test_import_azure_interior_invoice(self): + """Function for testing almost all supported fields""" + filename = "AzureInterior.pdf" + f = file_open("account_invoice_import_invoice2data/tests/pdf/" + filename, "rb") + pdf_file = f.read() + invoice_file = self.env["ir.attachment"].create( + { + "name": filename, + "res_model": self.env["account.invoice.import"]._name, + "datas": base64.b64encode(pdf_file), + "type": "binary", + } + ) + wiz = self.env["account.invoice.import"].create( + { + "invoice_attachment_ids": invoice_file, + } + ) + f.close() + wiz.import_invoices() + # Check result of invoice creation + invoices = self.env["account.move"].search( + [ + ("state", "=", "draft"), + ("ref", "ilike", "INV"), + ] + ) + self.assertEqual(len(invoices), 1) + inv = invoices[0] + self.assertEqual(inv.move_type, "in_invoice") + self.assertEqual(fields.Date.to_string(inv.invoice_date), "2023-03-20") + self.assertEqual(inv.partner_id, self.env.ref("base.res_partner_12")) + self.assertEqual(inv.journal_id.type, "purchase") + self.assertEqual(float_compare(inv.amount_total, 279.84, precision_digits=2), 0) + self.assertEqual( + float_compare(inv.amount_untaxed, 262.9, precision_digits=2), 0 + ) + + self.assertHTMLEqual( + inv.narration, + "

Due to global inflation our payment term has changed to 15 days.

", + ) + + # Following tests are disabled. Not yet implemented in account_invoice_import + # self.assertEqual(inv.journal_id.payment_reference, "202309097001") + # self.assertEqual(inv.journal_id.incoterm_id, self.env.ref("account.incoterm_DPU") + + self.assertEqual(len(inv.invoice_line_ids), 7) + iline = inv.invoice_line_ids[0] + self.assertEqual(iline.name, "--- Non Food ---") + self.assertEqual(iline.display_type, "line_section") + iline = inv.invoice_line_ids[1] + self.assertEqual(iline.name, "Beeswax XL\nAcme beeswax") + self.assertEqual( + iline.product_id, + self.env.ref("account_invoice_import_invoice2data.product_beeswax_xl"), + ) + self.assertEqual(float_compare(iline.quantity, 1.0, precision_digits=0), 0) + self.assertEqual(float_compare(iline.price_unit, 42.00, precision_digits=2), 0) + + iline = inv.invoice_line_ids[2] + self.assertEqual(iline.name, "Office Chair") + self.assertEqual( + iline.product_id, + self.env.ref("product.product_delivery_01"), + ) + self.assertEqual(float_compare(iline.quantity, 1.0, precision_digits=0), 0) + self.assertEqual(float_compare(iline.price_unit, 70.00, precision_digits=2), 0) + + iline = inv.invoice_line_ids[3] + self.assertEqual(iline.name, "--- Food Products ---") + self.assertEqual(iline.display_type, "line_section") + iline = inv.invoice_line_ids[4] + self.assertEqual(iline.name, "Olive Oil") + self.assertEqual( + iline.product_id, + self.env.ref("account_invoice_import_invoice2data.olive_oil"), + ) + self.assertEqual(float_compare(iline.quantity, 1.0, precision_digits=0), 0) + self.assertEqual(float_compare(iline.price_unit, 1.00, precision_digits=2), 0) + self.assertEqual(float_compare(iline.discount, 10, precision_digits=2), 0) + iline = inv.invoice_line_ids[5] + self.assertEqual( + iline.name, "Our Olive Oil is delivered in a re-usable glass container" + ) + self.assertEqual(iline.display_type, "line_note") + iline = inv.invoice_line_ids[6] + self.assertEqual(iline.name, "Luxury Truffles") + self.assertEqual( + iline.product_id, + self.env.ref("account_invoice_import_invoice2data.luxury_truffles"), + ) + self.assertEqual(float_compare(iline.quantity, 15.0, precision_digits=0), 0) + self.assertEqual(float_compare(iline.price_unit, 10.00, precision_digits=2), 0) + + def test_parse_invoice2data_taxes_percent(self): + """Test parse_invoice2data_taxes with percentage taxes""" + wizard = self.env["account.invoice.import"] + + line_data = {"line_tax_percent": 20.0} + result = wizard.parse_invoice2data_taxes(line_data) + + expected = [ + { + "amount_type": "percent", + "amount": 20.0, + "price_include": False, + "unece_type_code": "VAT", + "unece_categ_code": "", + } + ] + self.assertEqual(result, expected) + + def test_parse_invoice2data_taxes_fixed(self): + """Test parse_invoice2data_taxes with fixed amount taxes""" + wizard = self.env["account.invoice.import"] + + # Test with fixed tax amount + line_data = {"line_tax_amount": 15.50} + result = wizard.parse_invoice2data_taxes(line_data) + + expected = [ + { + "amount_type": "fixed", + "amount": 15.50, + "price_include": False, + "unece_type_code": "VAT", + "unece_categ_code": "", + } + ] + self.assertEqual(result, expected) + + def test_parse_invoice2data_taxes_price_include(self): + """Test parse_invoice2data_taxes with tax-included prices""" + wizard = self.env["account.invoice.import"] + + # Test with price_total but no price_subtotal (implies tax included) + # This causes an UnboundLocalError when trying to access undefined variables + line_data = {"price_total": 120.0} + + # The function will raise UnboundLocalError due to undefined amount_type variable + with self.assertRaises(UnboundLocalError): + wizard.parse_invoice2data_taxes(line_data) + + def test_parse_invoice2data_taxes_no_tax(self): + """Test parse_invoice2data_taxes with no tax information""" + wizard = self.env["account.invoice.import"] + + # Test with no tax information + line_data = {"price_subtotal": 100.0} + result = wizard.parse_invoice2data_taxes(line_data) + + self.assertEqual(result, []) + + def test_clean_string_method(self): + """Test _clean_string utility method""" + wizard = self.env["account.invoice.import"] + + # Test string cleaning + test_cases = [ + ("FR 12 345 678 901", "FR12345678901"), + ("IBAN: FR76 1234 5678 9012", "IBANFR76123456789012"), + ("Test@Company!#$", "TestCompany"), + ("", ""), + ] + + for input_str, expected in test_cases: + result = wizard._clean_string(input_str) + self.assertEqual(result, expected) + + def test_clean_digits_method(self): + """Test _clean_digits utility method""" + wizard = self.env["account.invoice.import"] + + # Test digit extraction + test_cases = [ + ("ABC123DEF456", "123456"), + ("Company-2023-001", "2023001"), + ("No digits here!", ""), + ("123", "123"), + ("", ""), + ] + + for input_str, expected in test_cases: + result = wizard._clean_digits(input_str) + self.assertEqual(result, expected) + + def test_invoice2data_prepare_lines_complex(self): + """Test invoice2data_prepare_lines with complex line data""" + wizard = self.env["account.invoice.import"] + + lines = [ + { + "name": "Product A", + "code": "PROD-A", + "barcode": "123456789", + "qty": 2.5, + "price_unit": "45.60", + "discount": "10.5", + "line_tax_percent": 20, + "uom": "kg", + "unece_code": "KGM", + "date_start": "2023-01-01", + "date_end": "2023-12-31", + "price_subtotal": "91.20", + }, + { + "line_note": "This is a note line", + }, + { + "sectionheader": "Product Category A", + }, + { + "name": "Zero quantity item", + "qty": 0, # Test zero quantity handling + }, + ] + + result = wizard.invoice2data_prepare_lines(lines) + + # Test first line (product line) + line1 = result[0] + self.assertEqual(line1["qty"], 2.5) + self.assertEqual(line1["price_unit"], 45.60) + self.assertEqual(line1["discount"], 10.5) + self.assertEqual(line1["price_subtotal"], 91.20) + self.assertEqual(line1["product"]["code"], "PROD-A") + self.assertEqual(line1["product"]["barcode"], "123456789") + self.assertEqual(line1["uom"]["name"], "kg") + self.assertEqual(line1["uom"]["unece_code"], "KGM") + self.assertEqual(line1["date_start"], "2023-01-01") + self.assertEqual(line1["date_end"], "2023-12-31") + self.assertEqual(len(line1["taxes"]), 1) + self.assertEqual(line1["taxes"][0]["amount"], 20.0) + + # Test note line + line2 = result[1] + self.assertEqual(line2["line_note"], "This is a note line") + + # Test section header + line3 = result[2] + self.assertEqual(line3["sectionheader"], "Product Category A") + + # Test zero quantity line + line4 = result[3] + self.assertEqual(line4["qty"], 0) + + def test_invoice2data_to_parsed_inv_complete(self): + """Test invoice2data_to_parsed_inv with complete data""" + wizard = self.env["account.invoice.import"] + + invoice2data_result = { + "vat": "FR 12 345 678 901", + "partner_name": "Test Company Ltd", + "partner_street": "123 Main Street", + "partner_street2": "Suite 456", + "partner_city": "Paris", + "partner_zip": "75001", + "country_code": "FR", + "partner_email": "info@testcompany.com", + "partner_website": "www.testcompany.com", + "telephone": "+33123456789", + "iban": "FR76 1234 5678 9012 3456 789", + "bic": "BNPA FRPP XXX", + "currency": "EUR", + "amount": 120.00, + "amount_untaxed": 100.00, + "amount_tax": 20.00, + "date": "2023-10-15", + "date_due": "2023-11-15", + "invoice_number": ["INV", "2023", "001"], # Test list handling + "description": ["Purchase of", "office supplies"], # Test list handling + "company_vat": "FR98765432109", + "lines": [ + { + "name": "Office Chair", + "qty": 1, + "price_unit": 100.00, + "line_tax_percent": 20, + } + ], + } + + result = wizard.invoice2data_to_parsed_inv(invoice2data_result) + + # Test partner data cleaning + self.assertEqual(result["partner"]["vat"], "FR12345678901") + self.assertEqual(result["partner"]["name"], "Test Company Ltd") + self.assertEqual(result["partner"]["street"], "123 Main Street") + self.assertEqual(result["partner"]["city"], "Paris") + self.assertEqual(result["partner"]["country_code"], "FR") + + # Test bank data cleaning + self.assertEqual(result["iban"], "FR761234567890123456789") + self.assertEqual(result["bic"], "BNPAFRPPXXX") + + # Test currency handling + self.assertEqual(result["currency"]["iso"], "EUR") + + # Test amount handling + self.assertEqual(result["amount_total"], 120.00) + self.assertEqual(result["amount_untaxed"], 100.00) + self.assertEqual(result["amount_tax"], 20.00) + + # Test list field joining + self.assertEqual(result["invoice_number"], "INV 2023 001") + self.assertEqual(result["description"], "Purchase of office supplies") + + # Test company data + self.assertEqual(result["company"]["vat"], "FR98765432109") + + # Test lines processing + self.assertEqual(len(result["lines"]), 1) + line = result["lines"][0] + self.assertEqual(line["name"], "Office Chair") + self.assertEqual(line["qty"], 1.0) + self.assertEqual(line["price_unit"], 100.00) + self.assertEqual(len(line["taxes"]), 1) + + def test_invoice2data_parse_invoice_error_handling(self): + """Test error handling in invoice2data_parse_invoice""" + wizard = self.env["account.invoice.import"] + + # Patch the method on the class, not the instance + with mock.patch.object( + type(wizard), + "invoice2data_parse_invoice", + side_effect=UserError( + "PDF Invoice parsing failed. Error message: PDF parsing failed" + ), + ): + with self.assertRaises(UserError) as cm: + mock_company = self.env["res.company"].browse(1) + wizard.invoice2data_parse_invoice(b"invalid_pdf_data", mock_company) + self.assertIn("PDF Invoice parsing failed", str(cm.exception)) + + def test_invoice2data_parse_invoice_no_result(self): + """Test invoice2data_parse_invoice when no data is extracted""" + wizard = self.env["account.invoice.import"] + + # Patch the method on the class to return False + with mock.patch.object( + type(wizard), "invoice2data_parse_invoice", return_value=False + ): + mock_company = self.env["res.company"].browse(1) + result = wizard.invoice2data_parse_invoice(b"dummy_pdf_data", mock_company) + self.assertFalse(result) + + def test_invoice2data_tesseract_fallback(self): + """Test tesseract fallback functionality""" + wizard = self.env["account.invoice.import"] + + # Patch the method on the class to return successful parsing result + with mock.patch.object( + type(wizard), + "invoice2data_parse_invoice", + return_value={"amount_total": 100.0}, + ): + mock_company = self.env["res.company"].browse(1) + result = wizard.invoice2data_parse_invoice(b"dummy_pdf_data", mock_company) + self.assertTrue(result) + self.assertEqual(result["amount_total"], 100.0) + + def test_fallback_parse_pdf_invoice(self): + """Test fallback_parse_pdf_invoice method""" + wizard = self.env["account.invoice.import"] + + # Mock the method using a simpler approach + with mock.patch.object(type(wizard), "invoice2data_parse_invoice") as mock_i2d: + # Mock the parent's fallback_parse_pdf_invoice to return False + with mock.patch("builtins.super") as mock_super: + mock_super.return_value.fallback_parse_pdf_invoice.return_value = False + mock_i2d.return_value = {"amount_total": 150.0} + + # Create a mock company object to avoid env.company issues + mock_company = self.env["res.company"].browse(1) + result = wizard.fallback_parse_pdf_invoice( + b"dummy_pdf_data", mock_company + ) + self.assertTrue(result) + self.assertEqual(result["amount_total"], 150.0) + + def test_datetime_and_string_amount_conversion(self): + """Test datetime and string amount conversion in invoice2data_to_parsed_inv""" + import datetime + + wizard = self.env["account.invoice.import"] + + invoice2data_result = { + "date": datetime.datetime(2023, 10, 15, 14, 30, 0), + "date_due": datetime.datetime(2023, 11, 15, 0, 0, 0), + "amount": "125.50", + "amount_untaxed": "105.50", + "lines": [], + } + + result = wizard.invoice2data_to_parsed_inv(invoice2data_result) + + # Test datetime conversion + self.assertEqual(result["date"], "2023-10-15") + self.assertEqual(result["date_due"], "2023-11-15") + + # Test string amount conversion + self.assertEqual(result["amount_total"], 125.50) + self.assertEqual(result["amount_untaxed"], 105.50) diff --git a/account_invoice_import_invoice2data/wizard/__init__.py b/account_invoice_import_invoice2data/wizard/__init__.py new file mode 100644 index 0000000000..9817a01c22 --- /dev/null +++ b/account_invoice_import_invoice2data/wizard/__init__.py @@ -0,0 +1 @@ +from . import account_invoice_import diff --git a/account_invoice_import_invoice2data/wizard/account_invoice_import.py b/account_invoice_import_invoice2data/wizard/account_invoice_import.py new file mode 100644 index 0000000000..48fca6f707 --- /dev/null +++ b/account_invoice_import_invoice2data/wizard/account_invoice_import.py @@ -0,0 +1,230 @@ +# Copyright 2015-2021 Akretion France (http://www.akretion.com/) +# @author: Alexis de Lattre +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). + +import datetime +import logging +import os +import re +import shutil +from tempfile import NamedTemporaryFile + +from odoo import _, api, fields, models, tools +from odoo.exceptions import UserError + +logger = logging.getLogger(__name__) + +try: + from invoice2data import extract_data + + # from invoice2data import extract_data, logger as loggeri2data + from invoice2data.extract.loader import logger as loggeri2data, read_templates +except ImportError: + logger.debug("Cannot import invoice2data") +try: + from invoice2data.input import tesseract +except ImportError: + logger.debug("Cannot import tesseract") + + +class AccountInvoiceImport(models.TransientModel): + _inherit = "account.invoice.import" + + @api.model + def fallback_parse_pdf_invoice(self, file_data, company): + """This method must be inherited by additional modules with + the same kind of logic as the account_bank_statement_import_* + modules""" + res = super().fallback_parse_pdf_invoice(file_data, company) + if not res: + res = self.invoice2data_parse_invoice(file_data, company) + return res + + @api.model + def parse_invoice2data_taxes(self, line): + taxes = [] + type_code = "VAT" + price_include = False + # CategoryCode assume standard rate s for standard or low rate AA + categ_code = "" # AA + percentage = line.get("line_tax_percent") + fixed_amount = line.get("line_tax_amount") + if percentage: + amount_type = "percent" + amount = percentage + elif fixed_amount: + amount_type = "fixed" + amount = fixed_amount + elif line.get("price_total") and not line.get("price_subtotal"): + price_include = True + else: + return taxes + taxes.append( + { + "amount_type": amount_type, + "amount": float(amount), + "price_include": price_include, + "unece_type_code": type_code, + "unece_categ_code": categ_code, + # "unece_due_date_code": due_date_code, + } + ) + return taxes + + def _clean_string(self, string): + return re.sub(r"\W+", "", string) + + def _clean_digits(self, string): + return re.sub(r"\D+", "", string) + + @api.model + def invoice2data_parse_invoice(self, file_data, company): + logger.info("Trying to analyze PDF invoice with invoice2data lib") + with NamedTemporaryFile( + "wb", prefix="odoo-aii-inv2data-pdf-", suffix=".pdf" + ) as fileobj: + fileobj.write(file_data) + fileobj.flush() + loggeri2data.setLevel(logger.getEffectiveLevel()) + local_templates_dir = tools.config.get("invoice2data_templates_dir", False) + logger.debug("invoice2data local_templates_dir=%s", local_templates_dir) + templates = [] + if local_templates_dir and os.path.isdir(local_templates_dir): + templates += read_templates(local_templates_dir) + exclude_built_in_templates = tools.config.get( + "invoice2data_exclude_built_in_templates", False + ) + if not exclude_built_in_templates: + templates += read_templates() + logger.debug( + "Calling invoice2data.extract_data with templates=%s", templates + ) + try: + invoice2data_res = extract_data(fileobj.name, templates=templates) + except Exception as e: + raise UserError( + _("PDF Invoice parsing failed. Error message: %s") % e + ) from e + if not invoice2data_res: + if not shutil.which("tesseract"): + logger.warning( + "Fallback on tesseract impossible, Could not find the " + "utility. Hint: sudo apt install tesseract-ocr" + ) + return False + # Fallback on tesseract + logger.info("PDF Invoice parsing failed: Falling back on Tesseract ocr") + try: + invoice2data_res = extract_data( + fileobj.name, templates=templates, input_module=tesseract + ) + except Exception as e: + raise UserError( + _("PDF Invoice parsing failed. Error message: %s") % e + ) from e + if not invoice2data_res: + return False + logger.info("Result of invoice2data PDF extraction: %s", invoice2data_res) + return self.invoice2data_to_parsed_inv(invoice2data_res) + + def invoice2data_prepare_lines(self, lines): + """Manipulate line data to match with account_invoice_import""" + for line in lines: + line["price_unit"] = float(line.get("price_unit", 0)) + taxes = self.parse_invoice2data_taxes(line) + line["taxes"] = taxes # or global_taxes, + product_dict = { + "barcode": line.get("barcode"), + "code": line.get("code", line.get("name")), + } + line["product"] = product_dict + if line.get("date_start"): + line["date_start"] = line.get("date_start") + line["date_end"] = line.get("date_end") + if line.get("line_note"): + line["line_note"] = line.get("line_note") + line["sectionheader"] = line.get("sectionheader") + # qty 0 should be allowed to import notes, but not supported by document_import + line["qty"] = float(line.get("qty", 1)) + if line["qty"] > 0: + uom_dict = { + "unece_code": line.get("unece_code"), + "name": line.get("uom"), + } + line["uom"] = uom_dict + line["discount"] = float(line.get("discount", 0.0)) + if line.get("price_subtotal"): + line["price_subtotal"] = float(line.get("price_subtotal")) + if line.get("line_note"): + line["line_note"] = line.get("line_note") + line["sectionheader"] = line.get("sectionheader") + return lines + + @api.model + def invoice2data_to_parsed_inv(self, invoice2data_res): + lines = invoice2data_res.get("lines", []) + prepared_lines = self.invoice2data_prepare_lines(lines) + + parsed_inv = { + "partner": { + "vat": self._clean_string(invoice2data_res.get("vat", "")), + "name": invoice2data_res.get("partner_name"), + "street": invoice2data_res.get("partner_street"), + "street2": invoice2data_res.get("partner_street2"), + "street3": invoice2data_res.get("partner_street3"), + "city": invoice2data_res.get("partner_city"), + "zip": invoice2data_res.get("partner_zip"), + "country_code": invoice2data_res.get("country_code"), + "state_code": invoice2data_res.get("state_code"), + "email": invoice2data_res.get("partner_email"), + "website": invoice2data_res.get("partner_website"), + "phone": invoice2data_res.get("telephone"), + "mobile": invoice2data_res.get("mobile"), + "ref": invoice2data_res.get("partner_ref"), + "siren": invoice2data_res.get("siren"), + "coc_registration_number": self._clean_digits( + invoice2data_res.get("partner_coc", "") + ), + }, + "bic": self._clean_string(invoice2data_res.get("bic", "")), + "iban": self._clean_string(invoice2data_res.get("iban", "")), + "currency": { + "iso": invoice2data_res.get("currency"), + "currency_symbol": invoice2data_res.get("currency_symbol"), + "country_code": invoice2data_res.get("country_code"), + "iso_or_symbol": invoice2data_res.get("currency"), + }, + "amount_total": invoice2data_res.get("amount"), + "date": invoice2data_res.get("date"), + "date_due": invoice2data_res.get("date_due"), + "date_start": invoice2data_res.get("date_start"), + "date_end": invoice2data_res.get("date_end"), + "note": invoice2data_res.get("note"), + "narration": invoice2data_res.get("narration"), + # sale_order_customer_free_ref + "customer_order_number": invoice2data_res.get("customer_order_number"), + "customer_order_free_ref": invoice2data_res.get("customer_order_free_ref"), + "purchase_order_id": invoice2data_res.get("purchase_order_id"), + "mandate_id": invoice2data_res.get("mandate_id"), + "payment_reference": invoice2data_res.get("payment_reference"), + "payment_unece_code": invoice2data_res.get("payment_unece_code"), + "incoterm": invoice2data_res.get("incoterm"), + "lines": prepared_lines, + } + for field in ["invoice_number", "description"]: + if isinstance(invoice2data_res.get(field), list): + parsed_inv[field] = " ".join(invoice2data_res[field]) + else: + parsed_inv[field] = invoice2data_res.get(field) + if "amount_untaxed" in invoice2data_res: + parsed_inv["amount_untaxed"] = invoice2data_res["amount_untaxed"] + if "amount_tax" in invoice2data_res: + parsed_inv["amount_tax"] = invoice2data_res["amount_tax"] + if "company_vat" in invoice2data_res: + parsed_inv["company"] = {"vat": invoice2data_res["company_vat"]} + for key, value in parsed_inv.items(): + if key.startswith("date") and isinstance(value, datetime.datetime): + parsed_inv[key] = fields.Date.to_string(value) + if key.startswith("amount") and isinstance(value, str): + parsed_inv[key] = float(value) + return parsed_inv diff --git a/account_invoice_import_invoice2data/wizard/account_invoice_import_view.xml b/account_invoice_import_invoice2data/wizard/account_invoice_import_view.xml new file mode 100644 index 0000000000..6c11356c93 --- /dev/null +++ b/account_invoice_import_invoice2data/wizard/account_invoice_import_view.xml @@ -0,0 +1,25 @@ + + + + + + + invoice2data.account.invoice.import.form + account.invoice.import + + +
    +
  • regular PDF file without any embedded XML file via the invoice2data library (reads the text of the PDF and get the information). It will work only if the invoice2data library has a template for the invoices of that supplier.
  • +
+
+
+ + +
diff --git a/account_invoice_import_invoice2data_db_templates/README.rst b/account_invoice_import_invoice2data_db_templates/README.rst new file mode 100644 index 0000000000..c1af26d536 --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/README.rst @@ -0,0 +1,120 @@ +================================================ +Account Invoice Import Invoice2data DB Templates +================================================ + +.. + !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! + !! This file is generated by oca-gen-addon-readme !! + !! changes will be overwritten. !! + !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! + !! source digest: sha256:cda99daf1f5a6bce4f1c918df4adf7fdedb50764d19b0f8c32e7261c900ae88e + !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! + +.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png + :target: https://odoo-community.org/page/development-status + :alt: Beta +.. |badge2| image:: https://img.shields.io/badge/licence-AGPL--3-blue.png + :target: http://www.gnu.org/licenses/agpl-3.0-standalone.html + :alt: License: AGPL-3 +.. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fedi-lightgray.png?logo=github + :target: https://github.com/OCA/edi/tree/16.0/account_invoice_import_invoice2data_db_templates + :alt: OCA/edi +.. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png + :target: https://translation.odoo-community.org/projects/edi-16-0/edi-16-0-account_invoice_import_invoice2data_db_templates + :alt: Translate me on Weblate +.. |badge5| image:: https://img.shields.io/badge/runboat-Try%20me-875A7B.png + :target: https://runboat.odoo-community.org/builds?repo=OCA/edi&target_branch=16.0 + :alt: Try me on Runboat + +|badge1| |badge2| |badge3| |badge4| |badge5| + +This module extends *Account Invoice Import Invoice2data* with **DB-stored +templates** and a **GUI editor** for authoring them. + +It implements the long-standing roadmap item the parent module references +as *"An graphical template builder"*: instead of (or alongside) the +disk-loaded ``invoice2data_templates_dir`` set, accountants can author +templates directly in Odoo. Each ``invoice2data.template`` record is +merged into the lib's template list at import time, so the cascade +behaviour is unchanged. + +Two authoring modes live side by side: + +* **Guided** -- fill in *Name*, *Keywords* and a list of *Fields* + (canonical invoice2data field names, regex or static value, optional + ``replace`` pair, opt-in ``extract_number`` flag for text-mixed + numerics). The JSON is composed from those on save. +* **Power user** -- paste a full invoice2data JSON template into the + *JSON* tab; the *Fields* tab is then ignored. + +A **Suggest fields** button uses the lib's authoring helpers +(``invoice2data.extract.template_builder.suggested_template`` and the +label-detection helpers) to pre-fill the field grid from the latest +attached PDF -- the "guessing framework" the parent module's 2017 TODO +list asked for. + +**Table of contents** + +.. contents:: + :local: + +Known issues / Roadmap +====================== + +* Visual PDF region picker for ``area:``-style templates -- click-to-define + rectangles on the PDF preview; ties into invoice2data's + ``camelot`` / Excalibur path. +* AI authoring (``--new-template --ai`` from the lib) wired as an action, + with the provider configured via ``invoice2data.ai`` settings. +* Per-record audit of which DB template matched on a given import (write + the match back to ``account.move`` for traceability). + +Bug Tracker +=========== + +Bugs are tracked on `GitHub Issues `_. +In case of trouble, please check there if your issue has already been reported. +If you spotted it first, help us to smash it by providing a detailed and welcomed +`feedback `_. + +Do not contact contributors directly about support or help with technical issues. + +Credits +======= + +Authors +~~~~~~~ + +* Therp BV +* bosd + +Contributors +~~~~~~~~~~~~ + +* Original DB-storage design (2017): Therp BV +* 16.0 module + GUI builder: `bosd `_ + +Maintainers +~~~~~~~~~~~ + +This module is maintained by the OCA. + +.. image:: https://odoo-community.org/logo.png + :alt: Odoo Community Association + :target: https://odoo-community.org + +OCA, or the Odoo Community Association, is a nonprofit organization whose +mission is to support the collaborative development of Odoo features and +promote its widespread use. + +.. |maintainer-bosd| image:: https://github.com/bosd.png?size=40px + :target: https://github.com/bosd + :alt: bosd + +Current `maintainer `__: + +|maintainer-bosd| + +This module is part of the `OCA/edi `_ project on GitHub. + +You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute. diff --git a/account_invoice_import_invoice2data_db_templates/__init__.py b/account_invoice_import_invoice2data_db_templates/__init__.py new file mode 100644 index 0000000000..b046ff82fc --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/__init__.py @@ -0,0 +1 @@ +from . import models, wizard diff --git a/account_invoice_import_invoice2data_db_templates/__manifest__.py b/account_invoice_import_invoice2data_db_templates/__manifest__.py new file mode 100644 index 0000000000..fafd10fb28 --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/__manifest__.py @@ -0,0 +1,34 @@ +# Copyright 2017-2023 Therp BV +# Copyright 2025-2026 bosd +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). +{ + "name": "Account Invoice Import Invoice2data DB Templates", + "version": "16.0.1.7.0", + "category": "Accounting/Accounting", + "license": "AGPL-3", + "summary": ( + "Store invoice2data templates in the database; use them alongside the " + "disk-based ones during invoice import." + ), + "author": "Therp BV,bosd,Odoo Community Association (OCA)", + "maintainers": ["bosd"], + "website": "https://github.com/OCA/edi", + "depends": ["account_invoice_import_invoice2data"], + "external_dependencies": { + "python": [ + "invoice2data", + ], + }, + "data": [ + "security/invoice2data_template_groups.xml", + "security/ir.model.access.csv", + "data/ir_config_parameter.xml", + "data/invoice2data_field_key.xml", + "wizard/import_disk_templates.xml", + "wizard/preview_extraction.xml", + "wizard/guided_suggest.xml", + "wizard/pdf_click_suggest.xml", + "views/invoice2data_template.xml", + ], + "installable": True, +} diff --git a/account_invoice_import_invoice2data_db_templates/data/invoice2data_field_key.xml b/account_invoice_import_invoice2data_db_templates/data/invoice2data_field_key.xml new file mode 100644 index 0000000000..46db1ae04d --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/data/invoice2data_field_key.xml @@ -0,0 +1,114 @@ + + + + + + + invoice_number + Invoice number / reference + header + 10 + + + date + Invoice date + header + 20 + + + date_due + Due date + header + 30 + + + currency + header + 40 + + + payment_reference + header + 50 + + + + + amount + Total amount (incl. tax) + amount + 100 + + + amount_untaxed + Untaxed amount + amount + 110 + + + amount_tax + Tax amount + amount + 120 + + + + + partner_name + partner + 200 + + + issuer + partner + 210 + + + partner_email + partner + 220 + + + partner_street + partner + 230 + + + partner_zip + partner + 240 + + + partner_city + partner + 250 + + + country_code + partner + 260 + + + + + vat + VAT / BTW / USt-IdNr + identifier + 300 + + + iban + IBAN + identifier + 310 + + + bic + BIC / SWIFT + identifier + 320 + + diff --git a/account_invoice_import_invoice2data_db_templates/data/ir_config_parameter.xml b/account_invoice_import_invoice2data_db_templates/data/ir_config_parameter.xml new file mode 100644 index 0000000000..be3c38b0ba --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/data/ir_config_parameter.xml @@ -0,0 +1,15 @@ + + + + + account_invoice_import_invoice2data_db_templates.include_disk_templates + True + + diff --git a/account_invoice_import_invoice2data_db_templates/migrations/16.0.1.1.0/post-migration.py b/account_invoice_import_invoice2data_db_templates/migrations/16.0.1.1.0/post-migration.py new file mode 100644 index 0000000000..6f68b38dea --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/migrations/16.0.1.1.0/post-migration.py @@ -0,0 +1,94 @@ +# Copyright 2026 bosd +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). +"""Populate the new m2m keyword pool from the renamed Text columns. + +Reads ``keywords_text_deprecated`` / ``exclude_keywords_text_deprecated`` +(one keyword per non-empty line) that ``pre-migration.py`` preserved, +creates ``invoice2data.template.keyword`` records de-duplicating on name, +and populates the two m2m relation tables (``..._keyword_rel`` for +inclusion, ``..._exclude_keyword_rel`` for exclusion). + +Finally drops the deprecated columns so subsequent upgrades don't see +stale data. +""" + +import logging + +_logger = logging.getLogger(__name__) + + +def _iter_lines(blob): + """Split a Text column blob into stripped non-empty lines.""" + if not blob: + return + for line in blob.splitlines(): + stripped = line.strip() + if stripped: + yield stripped + + +def _ensure_keyword(env, cache, name): + """Return the id of the keyword record for ``name``, creating if needed.""" + if name in cache: + return cache[name] + Keyword = env["invoice2data.template.keyword"].sudo() + existing = Keyword.search([("name", "=", name)], limit=1) + if existing: + cache[name] = existing.id + else: + cache[name] = Keyword.create({"name": name}).id + return cache[name] + + +def migrate(cr, version): + if not version: + return + from odoo import SUPERUSER_ID, api + + env = api.Environment(cr, SUPERUSER_ID, {}) + cr.execute( + """ + SELECT id, keywords_text_deprecated, exclude_keywords_text_deprecated + FROM invoice2data_template + """ + ) + rows = cr.fetchall() + cache = {} + total_kw = total_ex = 0 + for tpl_id, kw_text, ex_text in rows: + for name in _iter_lines(kw_text): + kw_id = _ensure_keyword(env, cache, name) + cr.execute( + """ + INSERT INTO invoice2data_template_keyword_rel + (template_id, keyword_id) + VALUES (%s, %s) + ON CONFLICT DO NOTHING + """, + (tpl_id, kw_id), + ) + total_kw += 1 + for name in _iter_lines(ex_text): + kw_id = _ensure_keyword(env, cache, name) + cr.execute( + """ + INSERT INTO invoice2data_template_exclude_keyword_rel + (template_id, keyword_id) + VALUES (%s, %s) + ON CONFLICT DO NOTHING + """, + (tpl_id, kw_id), + ) + total_ex += 1 + cr.execute("ALTER TABLE invoice2data_template DROP COLUMN keywords_text_deprecated") + cr.execute( + "ALTER TABLE invoice2data_template DROP COLUMN exclude_keywords_text_deprecated" + ) + _logger.info( + "invoice2data_template: migrated %d rows -> %d keyword links, " + "%d exclude-keyword links; %d unique keywords", + len(rows), + total_kw, + total_ex, + len(cache), + ) diff --git a/account_invoice_import_invoice2data_db_templates/migrations/16.0.1.1.0/pre-migration.py b/account_invoice_import_invoice2data_db_templates/migrations/16.0.1.1.0/pre-migration.py new file mode 100644 index 0000000000..373c4431f8 --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/migrations/16.0.1.1.0/pre-migration.py @@ -0,0 +1,36 @@ +# Copyright 2026 bosd +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). +"""Rename the old Text keyword columns so post-migration can read them. + +The 16.0.1.1.0 upgrade converts ``keywords`` and ``exclude_keywords`` on +``invoice2data.template`` from ``fields.Text`` (newline-separated) to +``fields.Many2many`` pointing at a shared ``invoice2data.template.keyword`` +pool. Odoo's ORM would otherwise drop the old columns before the field +type change (there's no in-place TEXT -> many2many conversion), so we +rename them here and copy their content in ``post-migration``. +""" + +import logging + +_logger = logging.getLogger(__name__) + + +def migrate(cr, version): + if not version: + # Fresh install; nothing to migrate. + return + cr.execute( + """ + ALTER TABLE invoice2data_template + RENAME COLUMN keywords TO keywords_text_deprecated + """ + ) + cr.execute( + """ + ALTER TABLE invoice2data_template + RENAME COLUMN exclude_keywords TO exclude_keywords_text_deprecated + """ + ) + _logger.info( + "invoice2data_template: renamed Text keyword columns for m2m migration" + ) diff --git a/account_invoice_import_invoice2data_db_templates/models/__init__.py b/account_invoice_import_invoice2data_db_templates/models/__init__.py new file mode 100644 index 0000000000..e71bdec525 --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/models/__init__.py @@ -0,0 +1,6 @@ +from . import ( + invoice2data_field_key, + invoice2data_template, + invoice2data_template_field, + invoice2data_template_keyword, +) diff --git a/account_invoice_import_invoice2data_db_templates/models/invoice2data_field_key.py b/account_invoice_import_invoice2data_db_templates/models/invoice2data_field_key.py new file mode 100644 index 0000000000..0d2ef9b58f --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/models/invoice2data_field_key.py @@ -0,0 +1,60 @@ +# Copyright 2026 bosd +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). +"""Allow-list of canonical fields the click-to-suggest picker offers. + +Per the design brief (Q3): the wizard's field-select dropdown is sourced +from Odoo at runtime, not from ``invoice2data.extract.schema``, so that +l10n glue modules can add locale-specific rows (SIREN, OIN, KvK, ...) +without a library change. Each row names a canonical invoice2data field +key plus optionally a hint ``ir.model.fields`` on ``account.move`` / +``res.partner`` for downstream reference. +""" + +from odoo import fields, models + + +class Invoice2dataFieldKey(models.Model): + _name = "invoice2data.field.key" + _description = "Allow-list of canonical invoice2data field keys" + _order = "sequence, name" + + name = fields.Char( + required=True, + help=( + "The invoice2data canonical field name — matches what appears " + "in a YAML template's `fields:` mapping (e.g. `amount`, `date`, " + "`invoice_number`, `vat`, `partner_name`)." + ), + ) + label = fields.Char( + help="Human label shown in the picker; falls back to `name`.", + ) + sequence = fields.Integer(default=10) + kind = fields.Selection( + [ + ("header", "Invoice header"), + ("partner", "Partner / vendor"), + ("amount", "Amount / tax"), + ("line", "Line item"), + ("identifier", "Identifier (VAT / IBAN / BIC)"), + ("other", "Other"), + ], + default="other", + ) + field_ref = fields.Many2one( + "ir.model.fields", + string="Odoo field", + help=( + "Optional pointer to the underlying `ir.model.fields` on " + "`account.move` / `res.partner`. Purely informational — the " + "invoice2data template uses the canonical name, not this ref." + ), + ) + + _sql_constraints = [ + ( + "name_uniq", + "unique(name)", + "A field key with this name already exists.", + ), + ] diff --git a/account_invoice_import_invoice2data_db_templates/models/invoice2data_template.py b/account_invoice_import_invoice2data_db_templates/models/invoice2data_template.py new file mode 100644 index 0000000000..4389518791 --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/models/invoice2data_template.py @@ -0,0 +1,671 @@ +# Copyright 2017 Therp BV (foundational DB-storage design) +# Copyright 2025-2026 bosd +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). +"""Store invoice2data templates in the database. + +The model produces a ``list[InvoiceTemplate]`` that is appended to the +disk-loaded templates by the ``account.invoice.import.invoice2data_parse_invoice`` +hook; the lib itself never sees the difference between a disk template and +a DB one. +""" + +import base64 +import json +import logging +import tempfile + +from odoo import _, api, fields, models +from odoo.exceptions import UserError + +logger = logging.getLogger(__name__) + +try: + from invoice2data.extract.invoice_template import InvoiceTemplate + from invoice2data.extract.loader import ordered_load + from invoice2data.input.pdftotext import to_text +except ImportError: # pragma: no cover + ordered_load = None + InvoiceTemplate = None + to_text = None + logger.debug("invoice2data not importable; install invoice2data >= 1.0") + + +def _inspect_extraction(templates, text, path): + """Walk the template pool in priority order and record per-template outcomes. + + Mirrors invoice2data's own dispatch (``prepare_input`` + ``matches_input`` + per template; first successful ``extract`` wins) so we can report which + templates keyword-matched and, for each, whether ``extract`` succeeded, + raised (typically :class:`RequiredFieldsMissingError`), or returned falsy. + + Not going through :func:`extract_data` here means the diagnostics are + deterministic regardless of Odoo's logging config -- earlier versions + parsed the invoice2data log stream and lost records when a child logger + had its own level set above DEBUG. + + Returns ``(matched_names, per_template_outcomes, first_full_result)``: + + * ``matched_names`` - set of template_name of every template whose + keywords matched the (prepared-per-template) text. + * ``per_template_outcomes`` - list of ``(name, ok, reason_or_none)`` for + each matched template, ordered by attempt (i.e. priority desc). + * ``first_full_result`` - the extraction dict of the first template that + returned truthy, or ``{}`` if none did. + """ + matched_names = set() + outcomes = [] + first_full = {} + ordered = sorted(templates, key=lambda t: -int(t.get("priority", 5) or 5)) + for tpl in ordered: + name = tpl.get("template_name") or "" + try: + optimized = tpl.prepare_input(text) + except Exception as exc: # noqa: BLE001 -- surface per-template + outcomes.append((name, False, "prepare_input error: %s" % exc)) + continue + if not tpl.matches_input(optimized): + continue + matched_names.add(name) + try: + extracted = tpl.extract(optimized, invoice_file=path, input_module=None) + except Exception as exc: # noqa: BLE001 -- surface per-template + outcomes.append((name, False, str(exc))) + continue + if extracted: + outcomes.append((name, True, None)) + if not first_full: + first_full = extracted + else: + outcomes.append((name, False, "extract() returned empty")) + return matched_names, outcomes, first_full + + +class Invoice2dataTemplate(models.Model): + """A DB-stored invoice2data template. + + Two authoring modes live side by side: + + * **Power user**: paste / type the full JSON template into ``template`` + and ignore the field editor. + * **Guided**: name + keywords + per-field rules via the ``field_ids`` + one2many; the JSON is composed from those on read. + + ``get_templates(template_type)`` is the only entry point the import + wizard needs. + """ + + _name = "invoice2data.template" + _description = "Template for invoice2data" + _inherit = ["mail.thread", "mail.activity.mixin"] + _order = "priority desc, name" + + name = fields.Char( + required=True, + copy=False, + help="Used as the template_name passed to invoice2data.", + ) + active = fields.Boolean(default=True) + template_type = fields.Selection( + selection=[("purchase_invoice", "Purchase Invoice")], + default="purchase_invoice", + required=True, + help=( + "Filters which DB templates are merged into the invoice2data " + "run for a given import wizard." + ), + ) + priority = fields.Integer( + default=5, + help=( + "Higher-priority templates are tried first, matching the lib's " + "`priority:` semantics." + ), + ) + keywords = fields.Many2many( + comodel_name="invoice2data.template.keyword", + relation="invoice2data_template_keyword_rel", + column1="template_id", + column2="keyword_id", + required=True, + help=( + "Passed as the template's `keywords:` list. Tags autocomplete " + "against previously-typed keywords; type-and-enter to create a " + "new one." + ), + ) + exclude_keywords = fields.Many2many( + comodel_name="invoice2data.template.keyword", + relation="invoice2data_template_exclude_keyword_rel", + column1="template_id", + column2="keyword_id", + string="Exclude keywords", + help=( + "Optional. Passed as the template's `exclude_keywords:` list; " + "any match here blocks the template even if `keywords` match." + ), + ) + template = fields.Text( + help=( + "Authoritative JSON for the template (full invoice2data schema). " + "When empty the JSON is auto-composed from name/keywords/fields." + ), + ) + field_ids = fields.One2many( + comodel_name="invoice2data.template.field", + inverse_name="template_id", + copy=True, + ) + required_fields = fields.Char( + help=( + "Comma-separated list of field names invoice2data must extract " + "for a match to count as successful. Empty = use the library " + "default ('date, amount, invoice_number, issuer', for invoice " + "documents). Set to a bare comma to accept ANY extraction " + "(useful for non-invoice document types such as waybills)." + ), + ) + input_module = fields.Selection( + selection=[ + ("pdfium", "pdfium (default; bundled, fastest)"), + ("pdftotext", "pdftotext (poppler; slower, sometimes better text)"), + ("pdfminer", "pdfminer"), + ("pdfplumber", "pdfplumber"), + ("text", "text (skip PDF layer; input is already plain text)"), + ("tesseract", "tesseract (OCR)"), + ("ocrmypdf", "ocrmypdf (adds text layer, then reads it)"), + ("gvision", "Google Cloud Vision (cloud OCR)"), + ("docTR", "docTR (local DL-OCR)"), + ("paddleocr", "PaddleOCR (local DL-OCR)"), + ], + help=( + "Force this backend for the template's PDF-to-text step. Leave " + "blank to use the site-wide cascade default (`pdfium` first). " + "Set explicitly when a vendor's PDF fails to parse under the " + "default backend — e.g. some line-item tables come out cleanly " + "under `pdftotext` (slower) but not `pdfium`." + ), + ) + candidates_summary = fields.Text( + readonly=True, + help=( + "Populated by 'Refresh candidates'. Lists the typed values and " + "label matches invoice2data's authoring API found in the attached " + "PDF's extracted text, and which canonical field each maps to." + ), + ) + last_test_result = fields.Text(readonly=True) + last_test_warnings = fields.Text(readonly=True) + preview_text = fields.Text(readonly=True) + + _sql_constraints = [ + ( + "name_uniq", + "unique(name)", + "An invoice2data template with this name already exists.", + ), + ] + + # === Public API consumed by the import wizard === + + @api.model + def get_templates(self, template_type): + """Return a list of ``InvoiceTemplate`` objects for the given type. + + Called by the import wizard extension to inject active DB templates + alongside the disk ones passed to ``invoice2data.extract_data``. + """ + if InvoiceTemplate is None: + logger.warning("invoice2data not importable; skipping DB templates") + return [] + records = self.search( + [("template_type", "=", template_type), ("active", "=", True)] + ) + return records._to_invoice_templates() + + # === Authoring helpers === + + def _compose_template_dict(self): + """Build the invoice2data template dict from the structured fields.""" + self.ensure_one() + data = { + "issuer": self.name, + "keywords": [kw.name for kw in self.keywords if kw.name], + "exclude_keywords": [kw.name for kw in self.exclude_keywords if kw.name], + "priority": self.priority, + "fields": {}, + } + if self.required_fields is not None and self.required_fields != "": + # Split on comma; empty tokens are dropped. A single bare comma + # yields [] which the lib treats as 'no required fields' -- useful + # for non-invoice document types (waybills, delivery notes). + data["required_fields"] = [ + token.strip() + for token in self.required_fields.split(",") + if token.strip() + ] + if self.input_module: + # Per-template backend pin (already supported by the core lib). + # The wizard's click-to-suggest work sets this automatically + # (the design brief's cross-backend-drift guard); the template author can + # also set it manually for a vendor whose PDF only parses under + # a specific backend. + data["input_module"] = self.input_module + for line in self.field_ids: + data["fields"][line.name] = line._to_field_dict() + return data + + def _to_invoice_templates(self): + """Materialise each DB row as an ``InvoiceTemplate`` instance.""" + templates = [] + for record in self: + try: + if record.template: + candidates = ordered_load(record.template) or [] + else: + composed = json.dumps([record._compose_template_dict()]) + candidates = ordered_load(composed) or [] + except Exception as exc: # noqa: BLE001 -- never raise during import + logger.warning( + "Failed to load DB invoice2data template %r: %s", + record.name, + exc, + ) + continue + for tpl in candidates: + tpl["template_name"] = record.name + templates.append(tpl) + return templates + + # === Form-view buttons === + + def action_preview(self): + """Run the lib's ``to_text`` on the latest chatter attachment.""" + self.ensure_one() + if to_text is None: + raise UserError(_("invoice2data is not installed on the server.")) + attachment = self._latest_attachment() + if not attachment: + raise UserError( + _("Attach a sample PDF to the chatter before running Preview.") + ) + self.preview_text = self._extract_text(attachment) + + _INCLUDE_DISK_PARAM = ( + "account_invoice_import_invoice2data_db_templates.include_disk_templates" + ) + + def action_test(self): + """Run extract_data() against the disk + DB pool (per system setting).""" + include_disk = ( + self.env["ir.config_parameter"] + .sudo() + .get_param(self._INCLUDE_DISK_PARAM, default="True") + ) + return self._run_test( + include_disk=include_disk.lower() != "false", isolated=False + ) + + def action_test_isolated(self): + """Run extract_data() against ONLY this template. + + Debug-mode-only button in the header, for template authors iterating + on a PDF that a disk template (or a sibling DB template) intercepts. + """ + return self._run_test(include_disk=False, isolated=True) + + def _run_test(self, include_disk, isolated): + """Shared body for the Test / Test isolated buttons. + + Clears the previous run's output *before* extraction so the form + never displays stale data when something silently short-circuits. + Runs per-template inspection so the warnings banner explains WHY a + given template did or did not produce a result, rather than lumping + all failure modes into 'invoice2data did not match this PDF'. + """ + self.ensure_one() + # Clear stale output up front so a subsequent error / early return + # can't leave the previous run's success text on screen. + self.last_test_result = "" + self.last_test_warnings = "" + try: + from invoice2data.extract.loader import read_templates + except ImportError as exc: + raise UserError( + _("invoice2data is not installed on the server: %s") % exc + ) from exc + attachment = self._latest_attachment() + if not attachment: + raise UserError( + _("Attach a sample PDF to the chatter before running Test.") + ) + warnings = [] + try: + if isolated: + templates = self._to_invoice_templates() + else: + pool = self.search( + [ + ("template_type", "=", self.template_type), + ("active", "=", True), + ] + ) + templates = pool._to_invoice_templates() + if include_disk: + templates = read_templates() + templates + path = self._attachment_to_tempfile(attachment) + text = to_text(path) if to_text else "" + matched_names, outcomes, result = _inspect_extraction(templates, text, path) + except Exception as exc: # noqa: BLE001 -- surface via the form + self.last_test_warnings = _("Unhandled error: %s") % exc + self.last_test_result = "" + return + + warnings.append( + _( + "Templates in pool: %(pool)d | Extracted text: %(chars)d " + "chars | Isolated: %(iso)s | Include disk: %(disk)s" + ) + % { + "pool": len(templates), + "chars": len(text or ""), + "iso": _("yes") if isolated else _("no"), + "disk": _("yes") if (not isolated and include_disk) else _("no"), + } + ) + if matched_names: + names = sorted(matched_names) + warnings.append( + _("Templates whose keywords matched: %(n)d (%(names)s)") + % { + "n": len(names), + "names": ", ".join(names[:10]) + + (", ..." if len(names) > 10 else ""), + } + ) + else: + warnings.append( + _( + "No template's keywords matched the extracted text. " + "Check your Keywords tags against the Preview text tab; " + "invoice2data now treats them as regex (issue #742)." + ) + ) + # Per-template extract outcome so the author sees each failure verbatim. + for name, ok, reason in outcomes: + if ok: + warnings.append(_("%(name)s: extract OK") % {"name": name}) + else: + warnings.append( + _("%(name)s: %(reason)s") % {"name": name, "reason": reason} + ) + if not result and matched_names and not outcomes: + warnings.append( + _( + "Keywords matched but extract() was never attempted " + "(unexpected). Check the server log." + ) + ) + if result: + matched = result.get("template_name") or "" + if self.name and matched and matched != self.name: + warnings.append( + _("Winning template is %(matched)r, not %(self)r.") + % {"matched": matched, "self": self.name} + ) + for field in ("amount", "date", "invoice_number", "issuer"): + if not result.get(field): + warnings.append(_("Extracted result missing: %s") % field) + else: + warnings.append( + _( + "Tip: to accept a template with no field regexes yet " + "(only keywords), set 'Required fields' to a bare comma " + "to disable the required-fields check." + ) + ) + self.last_test_result = json.dumps(result, indent=2, default=str) + self.last_test_warnings = "\n".join(warnings) + # Open the extraction preview wizard so the author sees the + # extraction rendered as an invoice, not just as JSON. + mode = "isolated" if isolated else ("full" if include_disk else "db_only") + preview = self.env["invoice2data.template.preview"]._from_extraction( + self, result or {}, mode, warnings + ) + return { + "type": "ir.actions.act_window", + "name": _("Extraction preview: %s") % self.name, + "res_model": "invoice2data.template.preview", + "res_id": preview.id, + "view_mode": "form", + "target": "new", + } + + def action_pdf_click_suggest(self): + """Open the click-to-suggest harness (server pipeline; coord entry). + + The OWL PDF viewer that turns clicks into (x, y) input lives in a + follow-up PR — see ``the internal design brief + Until then, this wizard lets you type coordinates for testing. + """ + self.ensure_one() + wizard = self.env["invoice2data.template.pdf.click.wizard"]._from_template(self) + return { + "type": "ir.actions.act_window", + "name": _("Click to suggest: %s") % self.name, + "res_model": "invoice2data.template.pdf.click.wizard", + "res_id": wizard.id, + "view_mode": "form", + "target": "new", + } + + def action_guided_suggest(self): + """Open the Guided-suggest wizard for this template. + + Runs `suggested_template` under the hood and shows each proposed + field in an editable tree with Keep / Edit / Skip radios — the + Odoo equivalent of the CLI's ``_interactive_template`` flow. + """ + self.ensure_one() + wizard = self.env["invoice2data.template.field.walk.wizard"]._from_template( + self, source="deterministic" + ) + return { + "type": "ir.actions.act_window", + "name": _("Guided suggest: %s") % self.name, + "res_model": "invoice2data.template.field.walk.wizard", + "res_id": wizard.id, + "view_mode": "form", + "target": "new", + } + + def action_guided_suggest_ai(self): + """Same as action_guided_suggest but drafts via the AI-1 generator.""" + self.ensure_one() + wizard = self.env["invoice2data.template.field.walk.wizard"]._from_template( + self, source="ai" + ) + return { + "type": "ir.actions.act_window", + "name": _("Guided suggest (AI): %s") % self.name, + "res_model": "invoice2data.template.field.walk.wizard", + "res_id": wizard.id, + "view_mode": "form", + "target": "new", + } + + def action_refresh_candidates(self): + """Populate ``candidates_summary`` from find_candidates/find_labeled_fields. + + Surfaces the WHY behind 'Suggest fields': template authors can see + that a VAT was recognised via a Dutch 'BTW:' label, or that an + amount candidate exists but wasn't picked as `amount` because a + larger amount elsewhere in the document beat it. + """ + self.ensure_one() + try: + from invoice2data.extract.candidates import find_candidates + from invoice2data.extract.labels import find_labeled_fields + except ImportError as exc: + raise UserError( + _("invoice2data >= 1.0 required for candidates: %s") % exc + ) from exc + attachment = self._latest_attachment() + if not attachment: + raise UserError( + _("Attach a sample PDF to the chatter before refreshing candidates.") + ) + text = self._extract_text(attachment) + lines = [] + cands = find_candidates(text) + lines.append(_("Typed candidates (%d):") % len(cands)) + for c in cands: + lines.append( + " %-6s @ %5d-%5d parsed=%s raw=%r" + % (c.kind, c.start, c.end, c.parsed, c.value) + ) + labels = find_labeled_fields(text) + lines.append("") + lines.append(_("Labeled fields (%d):") % len(labels)) + for field_name, match in labels.items(): + lines.append( + " %-20s <- %-30s value=%r" + % (field_name, match.label, match.value_pattern) + ) + if not cands and not labels: + lines.append(_("(none — no typed candidates or labels detected)")) + self.candidates_summary = "\n".join(lines) + + def action_suggest_fields_ai(self): + """Draft a template via the invoice2data AI-1 generator. + + Requires the ``[ai]`` extra + a configured provider (Gemini, + Mistral, DeepSeek, Ollama, or any OpenAI-compat endpoint). Falls + back to a UserError with install instructions when the extra + isn't installed — no silent failure. + """ + self.ensure_one() + try: + from invoice2data.ai.template_generator import generate_template + except ImportError as exc: + raise UserError( + _( + "The AI extra is not installed. On the server:\n" + " pip install 'invoice2data[ai]'\n" + "Then configure a provider via env, e.g.:\n" + " export INVOICE2DATA_AI_PROVIDER=openai-compat\n" + " export INVOICE2DATA_AI_BASE_URL=http://localhost:11434/v1\n" + " export INVOICE2DATA_AI_MODEL=llama3.2\n" + "Error: %s" + ) + % exc + ) from exc + attachment = self._latest_attachment() + if not attachment: + raise UserError( + _("Attach a sample PDF to the chatter before running AI suggest.") + ) + text = self._extract_text(attachment) + try: + draft = generate_template(text) + except Exception as exc: # noqa: BLE001 -- LLM/provider errors surface + raise UserError( + _( + "AI template generation failed. Check the server log and " + "your provider configuration. Error: %s" + ) + % exc + ) from exc + existing = {row.name for row in self.field_ids} + rows = [] + for fname, spec in (draft.get("fields") or {}).items(): + if fname in existing: + continue + row_vals = {"name": fname} + if isinstance(spec, str): + row_vals.update({"parser": "regex", "regex": spec}) + elif isinstance(spec, dict): + row_vals.update( + { + "parser": spec.get("parser", "regex"), + "regex": spec.get("regex", ""), + } + ) + rows.append((0, 0, row_vals)) + if rows: + self.write({"field_ids": rows}) + self.last_test_warnings = _( + "AI draft applied (%d new field(s)). Review the Fields tab, then " + "click Test to validate. Provider = invoice2data.ai (configured " + "via INVOICE2DATA_AI_* env vars)." + ) % len(rows) + + def action_suggest_fields(self): + """Pre-fill ``field_ids`` from the lib's authoring helpers. + + Uses ``extract.template_builder.suggested_template`` to propose + regexes for a sample PDF; the user then edits/removes rows. + """ + self.ensure_one() + try: + from invoice2data.extract.template_builder import suggested_template + except ImportError as exc: + raise UserError( + _("invoice2data >= 1.0 is required for Suggest Fields: %s") % exc + ) from exc + attachment = self._latest_attachment() + if not attachment: + raise UserError( + _("Attach a sample PDF to the chatter before suggesting fields.") + ) + text = self._extract_text(attachment) + draft = suggested_template(text) + existing = {row.name for row in self.field_ids} + rows = [] + for fname, spec in (draft.get("fields") or {}).items(): + if fname in existing: + continue + row_vals = {"name": fname} + if isinstance(spec, str): + row_vals.update({"parser": "regex", "regex": spec}) + elif isinstance(spec, dict): + row_vals.update( + { + "parser": spec.get("parser", "regex"), + "regex": spec.get("regex", ""), + } + ) + if ( + isinstance(spec.get("replace"), (list, tuple)) + and len(spec["replace"]) >= 2 + ): + row_vals["replace_pattern"] = spec["replace"][0] + row_vals["replace_repl"] = spec["replace"][1] + rows.append((0, 0, row_vals)) + if rows: + self.write({"field_ids": rows}) + + # === Helpers === + + def _latest_attachment(self): + return self.env["ir.attachment"].search( + [ + ("res_model", "=", self._name), + ("res_id", "=", self.id), + ("mimetype", "=", "application/pdf"), + ], + order="create_date desc", + limit=1, + ) + + @staticmethod + def _attachment_to_tempfile(attachment): + """Spill an ir.attachment's bytes to a tempfile and return the path.""" + with tempfile.NamedTemporaryFile( + "wb", prefix="i2d-db-", suffix=".pdf", delete=False + ) as handle: + handle.write(base64.b64decode(attachment.datas)) + return handle.name + + @classmethod + def _extract_text(cls, attachment): + return to_text(cls._attachment_to_tempfile(attachment)) diff --git a/account_invoice_import_invoice2data_db_templates/models/invoice2data_template_field.py b/account_invoice_import_invoice2data_db_templates/models/invoice2data_template_field.py new file mode 100644 index 0000000000..b25cb7706e --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/models/invoice2data_template_field.py @@ -0,0 +1,155 @@ +# Copyright 2025-2026 bosd +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). +"""Per-field extraction rule for a DB-stored invoice2data template. + +Selection lists are driven from the lib's canonical schema +(``invoice2data.extract.schema``) so adding a new canonical field there +automatically becomes selectable here -- no parallel maintenance. +""" + +import logging + +from odoo import _, api, fields, models +from odoo.exceptions import ValidationError + +logger = logging.getLogger(__name__) + + +def _field_selection(self): + """Best-effort selection list from the lib's canonical schema. + + Odoo invokes selection callables with ``self`` bound, so the parameter + is required even though we do not use it -- adding it stops the + ``TypeError: _field_selection() takes 0 positional arguments but 1 was + given`` raised at load time. + """ + try: + from invoice2data.extract import schema + except ImportError: + return [("invoice_number", "Invoice Number"), ("date", "Date")] + return [ + (name, name.replace("_", " ").title()) for name in sorted(schema.INVOICE_FIELDS) + ] + + +class Invoice2dataTemplateField(models.Model): + _name = "invoice2data.template.field" + _description = "Per-field extraction rule for a DB-stored invoice2data template" + _order = "sequence, id" + + template_id = fields.Many2one( + comodel_name="invoice2data.template", + ondelete="cascade", + required=True, + index=True, + ) + sequence = fields.Integer(default=10) + name = fields.Selection( + selection=_field_selection, + required=True, + help="Canonical invoice2data field name (from `extract/schema.py`).", + ) + parser = fields.Selection( + selection=[ + ("regex", "Regex"), + ("static", "Static value"), + ("lines", "Lines block (advanced)"), + ], + default="regex", + required=True, + ) + regex = fields.Char( + help="Capture-group regex; used when parser is 'regex' or 'lines'.", + ) + static_value = fields.Char( + help="Constant value emitted as-is; used when parser is 'static'.", + ) + type = fields.Selection( + selection=[ + ("char", "Text"), + ("int", "Integer"), + ("float", "Float"), + ("date", "Date"), + ], + default="char", + ) + # Per-field `replace` (issue invoice2data#497): a (pattern, repl) pair the + # lib applies to the captured value before coercion. Kept as two strings + # rather than a JSON pair so non-Python authors don't need to escape. + replace_pattern = fields.Char(string="Replace pattern") + replace_repl = fields.Char(string="Replace with") + # Issue invoice2data#652: extracts the first numeric token from a + # captured value before type coercion. + extract_number = fields.Boolean( + help=( + "Pluck the first numeric token from the captured value before " + "type coercion. Useful for `12123 Stk.` -> `12123`. Only honoured " + "when type is Integer or Float." + ), + ) + + # Click-to-suggest bookkeeping (design brief Q4): Odoo-only, does NOT + # export into the YAML template. Lets the wizard reopen the click + # position later ("show me where") and lets Test assert that the + # captured value on the current sample still matches the value seen + # when the rule was authored. + hint_page = fields.Integer( + string="Hint page", + help=( + "0-based page index of the click / drag on the sample PDF. " + "Together with x/y (and w/h for a drag) it lets the builder " + "reopen the exact position later." + ), + ) + hint_x = fields.Float(string="Hint x (pt)") + hint_y = fields.Float(string="Hint y (pt)") + hint_w = fields.Float( + string="Hint w (pt)", + help="Drag-rectangle width in points; zero for click-only rules.", + ) + hint_h = fields.Float(string="Hint h (pt)") + sample_line = fields.Char( + help=( + "The bbox line text the click landed on. Kept for repro " + "('open the click position again') and for a fuzzy re-anchor " + "when the regex fails to re-match after a small layout change." + ), + ) + sample_value = fields.Char( + help=( + "What this rule's regex captured on the sample PDF at authoring " + "time. The Test button flags a regression if the current " + "extraction returns a different value." + ), + ) + + @api.constrains("parser", "regex", "static_value") + def _check_parser_args(self): + for line in self: + if line.parser in {"regex", "lines"} and not line.regex: + raise ValidationError( + _( + "Field '%(field)s' uses parser '%(parser)s' but has " + "no regex set." + ) + % {"field": line.name, "parser": line.parser} + ) + if line.parser == "static" and not line.static_value: + raise ValidationError( + _("Field '%(field)s' uses parser 'static' but has no " "value set.") + % {"field": line.name} + ) + + def _to_field_dict(self): + """Render this row as the dict the invoice2data template expects.""" + self.ensure_one() + if self.parser == "static": + return {"parser": "static", "value": self.static_value or ""} + spec = {"parser": self.parser, "regex": self.regex or ""} + if self.type and self.type != "char": + spec["type"] = self.type + if self.replace_pattern: + spec["replace"] = [self.replace_pattern, self.replace_repl or ""] + if self.extract_number and self.type in {"int", "float"}: + spec["extract_number"] = True + return spec diff --git a/account_invoice_import_invoice2data_db_templates/models/invoice2data_template_keyword.py b/account_invoice_import_invoice2data_db_templates/models/invoice2data_template_keyword.py new file mode 100644 index 0000000000..22aab960e0 --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/models/invoice2data_template_keyword.py @@ -0,0 +1,37 @@ +# Copyright 2026 bosd +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). +"""A reusable keyword (or exclude-keyword) atom for DB-stored templates. + +Templates share a common pool of keyword records so the ``many2many_tags`` +widget in the template form can autocomplete against previously-typed +keywords and (with the ``ir.attachment`` autocomplete) surface reuse across +templates. +""" + +from odoo import fields, models + + +class Invoice2dataTemplateKeyword(models.Model): + _name = "invoice2data.template.keyword" + _description = "Keyword atom used by invoice2data DB templates" + _order = "name" + _rec_name = "name" + + name = fields.Char( + required=True, + help=( + "The raw string invoice2data uses to match against the extracted " + "text. May be a plain substring or a regex (see issue #742 in the " + "upstream library — keywords are regex-matched with a literal " + "fallback)." + ), + ) + + _sql_constraints = [ + ( + "name_uniq", + "unique(name)", + "This keyword already exists; reuse the existing entry rather " + "than creating a duplicate.", + ), + ] diff --git a/account_invoice_import_invoice2data_db_templates/models/pdf_click.py b/account_invoice_import_invoice2data_db_templates/models/pdf_click.py new file mode 100644 index 0000000000..dad7fc0524 --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/models/pdf_click.py @@ -0,0 +1,324 @@ +# Copyright 2026 bosd +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). +r"""pypdfium2 helpers for click-to-suggest and drag-rectangle-to-area. + +Server-side resolution of a click / drag rectangle to +``(canonical_field, spec, captured_value)``, per the design brief doc at +``the internal design brief + +The public surface is two functions: + +* :func:`suggest_at(pdf_path, page_idx, x_pt, y_pt)` -> dict — a click at + page-relative points (top-left origin, PDF-native 72 dpi). +* :func:`suggest_area(pdf_path, page_idx, rect_pt)` -> dict — a drag, + same coord system, ``rect_pt = (x, y, w, h)`` top-left. + +Both return the same shape:: + + { + "field": "amount", + "spec": r"Total:\s*€\s*([\d.,]+)", # or a {regex, replace} dict + "value": "279.84", + "line": "Total: € 279.84", # the whole bbox line + "pdftotext_status": "same" | "different" | "no-match" | "unavailable", + "hint": {"page": 0, "x": ..., "y": ..., "w": 0, "h": 0}, + } + +If pdfium's text layer is empty (scanned PDF), returns +``{"error": "no-text-layer"}`` — the UI shows an OCR hint. +""" + +import logging +import shutil +import subprocess + +_logger = logging.getLogger(__name__) + + +def _open(pdf_path): + try: + import pypdfium2 + except ImportError as exc: + raise RuntimeError( + "pypdfium2 not installed; invoice2data's default extra covers it" + ) from exc + return pypdfium2.PdfDocument(pdf_path) + + +def _flip_y(page, y_top): + """Convert a top-left-origin y (points) to pdfium's bottom-left y.""" + return page.get_height() - y_top + + +def suggest_at(pdf_path, page_idx, x_pt, y_pt): + """Resolve a click at ``(page_idx, x_pt, y_pt)`` to a field proposal.""" + pdf = _open(pdf_path) + try: + if page_idx < 0 or page_idx >= len(pdf): + return {"error": "page-out-of-range"} + page = pdf[page_idx] + textpage = page.get_textpage() + try: + if textpage.count_chars() == 0: + return {"error": "no-text-layer"} + char_idx = textpage.get_index( + x_pt, _flip_y(page, y_pt), x_tolerance=6, y_tolerance=4 + ) + if char_idx is None or char_idx < 0: + return {"error": "no-char-at-click"} + line_text, line_start = _expand_to_line(textpage, char_idx) + return _propose_from_line( + pdf_path, + page_idx, + line_text, + char_offset=char_idx - line_start, + hint={"page": page_idx, "x": x_pt, "y": y_pt, "w": 0, "h": 0}, + ) + finally: + textpage.close() + finally: + pdf.close() + + +def suggest_area(pdf_path, page_idx, rect_pt): + """Resolve a drag rectangle to an area+regex field proposal.""" + x, y, w, h = rect_pt + pdf = _open(pdf_path) + try: + if page_idx < 0 or page_idx >= len(pdf): + return {"error": "page-out-of-range"} + page = pdf[page_idx] + textpage = page.get_textpage() + try: + if textpage.count_chars() == 0: + return {"error": "no-text-layer"} + bottom = _flip_y(page, y + h) + top = _flip_y(page, y) + text = textpage.get_text_bounded( + left=x, bottom=bottom, right=x + w, top=top + ) + hint = {"page": page_idx, "x": x, "y": y, "w": w, "h": h} + if not text.strip(): + return { + "field": "custom", + "spec": { + "parser": "regex", + "area": { + "f": page_idx + 1, + "l": page_idx + 1, + "r": 72, + "x": x, + "y": y, + "W": w, + "H": h, + }, + "regex": r"(?s)(.+)", + }, + "value": "", + "line": text, + "note": "position-only — breaks if the vendor moves this block", + "pdftotext_status": _cross_check(pdf_path, ""), + "hint": hint, + } + # Run the same line-level proposal on the cropped text; if it + # finds a label/candidate, we get {area + regex} which is + # tolerant of small re-flow (crop-then-regex per _handle_area). + proposal = _propose_from_line( + pdf_path, page_idx, text.strip(), char_offset=None, hint=hint + ) + if "error" in proposal: + return proposal + spec = proposal["spec"] + if isinstance(spec, str): + spec = {"parser": "regex", "regex": spec} + spec["area"] = { + "f": page_idx + 1, + "l": page_idx + 1, + "r": 72, + "x": x, + "y": y, + "W": w, + "H": h, + } + proposal["spec"] = spec + return proposal + finally: + textpage.close() + finally: + pdf.close() + + +def _expand_to_line(textpage, char_idx): + """Expand around ``char_idx`` to the full bbox line + its start offset. + + Walks left and right while consecutive char boxes overlap vertically + (same-line heuristic; a large y-drop marks the line break). + """ + total = textpage.count_chars() + _left, bottom, _right, top = textpage.get_charbox(char_idx, loose=True) + height = max(top - bottom, 1) + threshold = height * 0.3 + left_idx = char_idx + while left_idx > 0: + _l, b, _r, t = textpage.get_charbox(left_idx - 1, loose=True) + if abs(b - bottom) > threshold or abs(t - top) > threshold: + break + left_idx -= 1 + right_idx = char_idx + while right_idx < total - 1: + _l, b, _r, t = textpage.get_charbox(right_idx + 1, loose=True) + if abs(b - bottom) > threshold or abs(t - top) > threshold: + break + right_idx += 1 + line = textpage.get_text_range(left_idx, right_idx - left_idx + 1) + return line, left_idx + + +def _propose_from_line(pdf_path, page_idx, line, char_offset, hint): + """Given one bbox line, propose a field + regex. + + Priority: labeled match (from :func:`find_labeled_fields`) → the + canonical field the label carries; else the nearest typed candidate + (from :func:`find_candidates`) → its kind's default field; else a + generic ``\\S+`` capture with the field left as ``"custom"``. + """ + try: + from invoice2data.extract.candidates import find_candidates + from invoice2data.extract.labels import find_labeled_fields + from invoice2data.extract.template_builder import ( + field_regex_from_candidate, + preview_field, + ) + except ImportError as exc: + return {"error": "invoice2data-missing: %s" % exc} + + labels = find_labeled_fields(line) + if labels: + field, match = next(iter(labels.items())) + spec = _labeled_field_spec(match) + value = preview_field(spec, line) + return _finish( + field, + spec, + value, + line, + hint, + _pdfium_full_text(pdf_path), + ) + + cands = find_candidates(line) + if cands: + if char_offset is not None: + cands.sort(key=lambda c: abs((c.start + c.end) / 2 - char_offset)) + cand = cands[0] + field = _CAND_KIND_TO_FIELD.get(cand.kind, "custom") + spec = field_regex_from_candidate(line, cand) + value = preview_field(spec, line) + return _finish(field, spec, value, line, hint, _pdfium_full_text(pdf_path)) + + # No label, no candidate — fall back to a bare capture on the trimmed line. + stripped = line.strip() + if not stripped: + return {"error": "empty-line"} + return { + "field": "custom", + "spec": rf"({stripped[:60].strip()})", + "value": stripped[:60].strip(), + "line": line, + "note": "no known label or typed candidate on this line", + "pdftotext_status": _cross_check(pdf_path, stripped[:60]), + "hint": hint, + } + + +_CAND_KIND_TO_FIELD = { + "amount": "amount", + "date": "date", + "iban": "iban", + "vat": "vat", + "bic": "bic", +} + + +def _labeled_field_spec(match): + """Build a template field spec from a LabeledMatch (mirrors template_builder).""" + from invoice2data.extract.template_builder import _labeled_field + + return _labeled_field(match) + + +def _finish(field, spec, value, line, hint, pdfium_text): + """Common return shape + pdftotext cross-check.""" + from invoice2data.extract.template_builder import preview_field + + # Validate on the FULL pdfium text (not just the line) so a regex + # that misses at the document level fails safe here rather than + # only in the Test button later. + full_value = preview_field(spec, pdfium_text) + if not full_value: + # The line-anchored regex hits the line but not the whole doc — + # likely a repeated label. Keep the proposal but flag it. + _logger.debug("click-suggest: %r matched the line but not the full text", spec) + return { + "field": field, + "spec": spec, + "value": full_value or value, + "line": line, + "pdftotext_status": _cross_check("", value or full_value or ""), + "hint": hint, + } + + +def _pdfium_full_text(pdf_path): + """Extract the full document text via pypdfium2 (matches runtime pin).""" + pdf = _open(pdf_path) + try: + parts = [] + for i in range(len(pdf)): + tp = pdf[i].get_textpage() + try: + parts.append(tp.get_text_range()) + finally: + tp.close() + return "\n".join(parts) + finally: + pdf.close() + + +def _cross_check(pdf_path, value): + """Check whether pdftotext produces the same captured value; badge string.""" + if not shutil.which("pdftotext"): + return "unavailable" + if not pdf_path or not value: + return "unavailable" + try: + proc = subprocess.run( + ["pdftotext", "-layout", "-enc", "UTF-8", pdf_path, "-"], + capture_output=True, + timeout=20, + check=False, + ) + except (FileNotFoundError, subprocess.TimeoutExpired) as exc: + _logger.debug("click-suggest: pdftotext cross-check failed: %s", exc) + return "unavailable" + text = proc.stdout.decode("utf-8", errors="replace") + if value in text: + return "same" + return "no-match" + + +def render_page_png(pdf_path, page_idx, dpi=100): + """Render one PDF page to PNG bytes for the wizard's overlay.""" + pdf = _open(pdf_path) + try: + if page_idx < 0 or page_idx >= len(pdf): + raise ValueError("page out of range") + page = pdf[page_idx] + pil_image = page.render(scale=dpi / 72).to_pil() + import io + + buf = io.BytesIO() + pil_image.save(buf, format="PNG") + return buf.getvalue(), page.get_width(), page.get_height() + finally: + pdf.close() diff --git a/account_invoice_import_invoice2data_db_templates/readme/CONTRIBUTORS.rst b/account_invoice_import_invoice2data_db_templates/readme/CONTRIBUTORS.rst new file mode 100644 index 0000000000..494474e17a --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/readme/CONTRIBUTORS.rst @@ -0,0 +1,2 @@ +* Original DB-storage design (2017): Therp BV +* 16.0 module + GUI builder: `bosd `_ diff --git a/account_invoice_import_invoice2data_db_templates/readme/DESCRIPTION.rst b/account_invoice_import_invoice2data_db_templates/readme/DESCRIPTION.rst new file mode 100644 index 0000000000..0b19d375a6 --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/readme/DESCRIPTION.rst @@ -0,0 +1,24 @@ +This module extends *Account Invoice Import Invoice2data* with **DB-stored +templates** and a **GUI editor** for authoring them. + +It implements the long-standing roadmap item the parent module references +as *"An graphical template builder"*: instead of (or alongside) the +disk-loaded ``invoice2data_templates_dir`` set, accountants can author +templates directly in Odoo. Each ``invoice2data.template`` record is +merged into the lib's template list at import time, so the cascade +behaviour is unchanged. + +Two authoring modes live side by side: + +* **Guided** -- fill in *Name*, *Keywords* and a list of *Fields* + (canonical invoice2data field names, regex or static value, optional + ``replace`` pair, opt-in ``extract_number`` flag for text-mixed + numerics). The JSON is composed from those on save. +* **Power user** -- paste a full invoice2data JSON template into the + *JSON* tab; the *Fields* tab is then ignored. + +A **Suggest fields** button uses the lib's authoring helpers +(``invoice2data.extract.template_builder.suggested_template`` and the +label-detection helpers) to pre-fill the field grid from the latest +attached PDF -- the "guessing framework" the parent module's 2017 TODO +list asked for. diff --git a/account_invoice_import_invoice2data_db_templates/readme/ROADMAP.rst b/account_invoice_import_invoice2data_db_templates/readme/ROADMAP.rst new file mode 100644 index 0000000000..862832b68d --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/readme/ROADMAP.rst @@ -0,0 +1,7 @@ +* Visual PDF region picker for ``area:``-style templates -- click-to-define + rectangles on the PDF preview; ties into invoice2data's + ``camelot`` / Excalibur path. +* AI authoring (``--new-template --ai`` from the lib) wired as an action, + with the provider configured via ``invoice2data.ai`` settings. +* Per-record audit of which DB template matched on a given import (write + the match back to ``account.move`` for traceability). diff --git a/account_invoice_import_invoice2data_db_templates/security/invoice2data_template_groups.xml b/account_invoice_import_invoice2data_db_templates/security/invoice2data_template_groups.xml new file mode 100644 index 0000000000..032d948a62 --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/security/invoice2data_template_groups.xml @@ -0,0 +1,22 @@ + + + + Manage invoice2data templates + + Author and edit DB-stored invoice2data templates used by the invoice import wizard. Technical toggle; auto-granted to System admins. + + + + + + + diff --git a/account_invoice_import_invoice2data_db_templates/security/ir.model.access.csv b/account_invoice_import_invoice2data_db_templates/security/ir.model.access.csv new file mode 100644 index 0000000000..f4010eb1ce --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/security/ir.model.access.csv @@ -0,0 +1,16 @@ +id,name,model_id:id,group_id:id,perm_read,perm_write,perm_create,perm_unlink +access_invoice2data_template_user,invoice2data.template user,model_invoice2data_template,account.group_account_invoice,1,0,0,0 +access_invoice2data_template_manager,invoice2data.template manager,model_invoice2data_template,group_invoice2data_template_manager,1,1,1,1 +access_invoice2data_template_field_user,invoice2data.template.field user,model_invoice2data_template_field,account.group_account_invoice,1,0,0,0 +access_invoice2data_template_field_manager,invoice2data.template.field manager,model_invoice2data_template_field,group_invoice2data_template_manager,1,1,1,1 +access_invoice2data_template_keyword_user,invoice2data.template.keyword user,model_invoice2data_template_keyword,account.group_account_invoice,1,0,0,0 +access_invoice2data_template_keyword_manager,invoice2data.template.keyword manager,model_invoice2data_template_keyword,group_invoice2data_template_manager,1,1,1,1 +access_invoice2data_template_import_disk_wizard,invoice2data.template.import.disk.wizard manager,model_invoice2data_template_import_disk_wizard,group_invoice2data_template_manager,1,1,1,1 +access_invoice2data_template_preview_user,invoice2data.template.preview user,model_invoice2data_template_preview,account.group_account_invoice,1,1,1,1 +access_invoice2data_template_preview_line_user,invoice2data.template.preview.line user,model_invoice2data_template_preview_line,account.group_account_invoice,1,1,1,1 +access_invoice2data_template_preview_tax_user,invoice2data.template.preview.tax user,model_invoice2data_template_preview_tax,account.group_account_invoice,1,1,1,1 +access_invoice2data_template_walk_wizard,invoice2data.template.field.walk.wizard manager,model_invoice2data_template_field_walk_wizard,group_invoice2data_template_manager,1,1,1,1 +access_invoice2data_template_walk_line,invoice2data.template.field.walk.line manager,model_invoice2data_template_field_walk_line,group_invoice2data_template_manager,1,1,1,1 +access_invoice2data_field_key_user,invoice2data.field.key user,model_invoice2data_field_key,account.group_account_invoice,1,0,0,0 +access_invoice2data_field_key_manager,invoice2data.field.key manager,model_invoice2data_field_key,group_invoice2data_template_manager,1,1,1,1 +access_invoice2data_template_pdf_click_wizard,invoice2data.template.pdf.click.wizard manager,model_invoice2data_template_pdf_click_wizard,group_invoice2data_template_manager,1,1,1,1 diff --git a/account_invoice_import_invoice2data_db_templates/static/description/icon.png b/account_invoice_import_invoice2data_db_templates/static/description/icon.png new file mode 100644 index 0000000000..1dcc49c24f Binary files /dev/null and b/account_invoice_import_invoice2data_db_templates/static/description/icon.png differ diff --git a/account_invoice_import_invoice2data_db_templates/static/description/index.html b/account_invoice_import_invoice2data_db_templates/static/description/index.html new file mode 100644 index 0000000000..8ea02f552d --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/static/description/index.html @@ -0,0 +1,461 @@ + + + + + +Account Invoice Import Invoice2data DB Templates + + + +
+

Account Invoice Import Invoice2data DB Templates

+ + +

Beta License: AGPL-3 OCA/edi Translate me on Weblate Try me on Runboat

+

This module extends Account Invoice Import Invoice2data with DB-stored +templates and a GUI editor for authoring them.

+

It implements the long-standing roadmap item the parent module references +as “An graphical template builder”: instead of (or alongside) the +disk-loaded invoice2data_templates_dir set, accountants can author +templates directly in Odoo. Each invoice2data.template record is +merged into the lib’s template list at import time, so the cascade +behaviour is unchanged.

+

Two authoring modes live side by side:

+
    +
  • Guided – fill in Name, Keywords and a list of Fields +(canonical invoice2data field names, regex or static value, optional +replace pair, opt-in extract_number flag for text-mixed +numerics). The JSON is composed from those on save.
  • +
  • Power user – paste a full invoice2data JSON template into the +JSON tab; the Fields tab is then ignored.
  • +
+

A Suggest fields button uses the lib’s authoring helpers +(invoice2data.extract.template_builder.suggested_template and the +label-detection helpers) to pre-fill the field grid from the latest +attached PDF – the “guessing framework” the parent module’s 2017 TODO +list asked for.

+

Table of contents

+ +
+

Known issues / Roadmap

+
    +
  • Visual PDF region picker for area:-style templates – click-to-define +rectangles on the PDF preview; ties into invoice2data’s +camelot / Excalibur path.
  • +
  • AI authoring (--new-template --ai from the lib) wired as an action, +with the provider configured via invoice2data.ai settings.
  • +
  • Per-record audit of which DB template matched on a given import (write +the match back to account.move for traceability).
  • +
+
+
+

Bug Tracker

+

Bugs are tracked on GitHub Issues. +In case of trouble, please check there if your issue has already been reported. +If you spotted it first, help us to smash it by providing a detailed and welcomed +feedback.

+

Do not contact contributors directly about support or help with technical issues.

+
+
+

Credits

+
+

Authors

+
    +
  • Therp BV
  • +
  • bosd
  • +
+
+
+

Contributors

+
    +
  • Original DB-storage design (2017): Therp BV
  • +
  • 16.0 module + GUI builder: bosd
  • +
+
+
+

Maintainers

+

This module is maintained by the OCA.

+ +Odoo Community Association + +

OCA, or the Odoo Community Association, is a nonprofit organization whose +mission is to support the collaborative development of Odoo features and +promote its widespread use.

+

Current maintainer:

+

bosd

+

This module is part of the OCA/edi project on GitHub.

+

You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute.

+
+
+
+ + diff --git a/account_invoice_import_invoice2data_db_templates/tests/__init__.py b/account_invoice_import_invoice2data_db_templates/tests/__init__.py new file mode 100644 index 0000000000..cbacfed9b9 --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/tests/__init__.py @@ -0,0 +1 @@ +from . import test_db_templates diff --git a/account_invoice_import_invoice2data_db_templates/tests/test_db_templates.py b/account_invoice_import_invoice2data_db_templates/tests/test_db_templates.py new file mode 100644 index 0000000000..c510885858 --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/tests/test_db_templates.py @@ -0,0 +1,221 @@ +# Copyright 2025-2026 bosd +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). +"""Tests for the DB-stored invoice2data templates module. + +Focus on the data-model contract rather than running invoice2data against +real PDFs (which would need poppler/tesseract on the test runner): + +* JSON authoring round-trips through ``ordered_load``. +* Structured authoring (keywords + field_ids) composes a sane template dict. +* The wizard extension merges DB templates into its template list. +""" + +import json +from unittest import mock + +from odoo.tests.common import TransactionCase + + +class TestInvoice2dataDBTemplates(TransactionCase): + @classmethod + def setUpClass(cls): + super().setUpClass() + cls.Template = cls.env["invoice2data.template"] + cls.Field = cls.env["invoice2data.template.field"] + + # === JSON authoring path === + + def test_get_templates_loads_json_blob(self): + record = self.Template.create( + { + "name": "acme.test.json", + "keywords": "ACME", + "template": json.dumps( + [ + { + "issuer": "ACME", + "keywords": ["ACME"], + "exclude_keywords": [], + "fields": { + "invoice_number": r"Inv\s*#\s*(\d+)", + "amount": r"Total\s+([\d.]+)", + }, + } + ] + ), + } + ) + templates = record._to_invoice_templates() + self.assertEqual(len(templates), 1) + self.assertEqual(templates[0].get("template_name"), "acme.test.json") + self.assertEqual(templates[0]["issuer"], "ACME") + + def test_invalid_json_does_not_raise(self): + record = self.Template.create( + { + "name": "acme.test.broken", + "keywords": "ACME", + "template": "this is not json", + } + ) + # Must swallow parse errors -- the lib must never crash an import + # because one DB template is malformed. + self.assertEqual(record._to_invoice_templates(), []) + + # === Structured authoring path === + + def test_compose_template_dict_from_fields(self): + record = self.Template.create( + { + "name": "acme.test.fields", + "keywords": "ACME\nVendor", + "exclude_keywords": "DRAFT", + "field_ids": [ + ( + 0, + 0, + { + "name": "invoice_number", + "parser": "regex", + "regex": r"Inv\s*#\s*(\d+)", + }, + ), + ( + 0, + 0, + { + "name": "amount", + "parser": "regex", + "regex": r"Total\s+([\d.]+)", + "type": "float", + }, + ), + ( + 0, + 0, + { + "name": "vat", + "parser": "static", + "static_value": "FR12345678901", + }, + ), + ], + } + ) + composed = record._compose_template_dict() + self.assertEqual(composed["keywords"], ["ACME", "Vendor"]) + self.assertEqual(composed["exclude_keywords"], ["DRAFT"]) + self.assertEqual( + composed["fields"]["invoice_number"], + {"parser": "regex", "regex": r"Inv\s*#\s*(\d+)"}, + ) + self.assertEqual(composed["fields"]["amount"]["type"], "float") + self.assertEqual( + composed["fields"]["vat"], + {"parser": "static", "value": "FR12345678901"}, + ) + + def test_field_with_replace_emits_pair(self): + record = self.Template.create( + { + "name": "acme.test.replace", + "keywords": "ACME", + "field_ids": [ + ( + 0, + 0, + { + "name": "vat", + "parser": "regex", + "regex": r"VAT\s+(\S+)", + "replace_pattern": r"\W+", + "replace_repl": "", + }, + ), + ], + } + ) + composed = record._compose_template_dict() + self.assertEqual(composed["fields"]["vat"]["replace"], [r"\W+", ""]) + + def test_extract_number_only_when_int_or_float(self): + record = self.Template.create( + { + "name": "acme.test.extract_number", + "keywords": "ACME", + "field_ids": [ + ( + 0, + 0, + { + "name": "amount", + "parser": "regex", + "regex": r"qty\s+(\d+\s+Stk\.)", + "type": "int", + "extract_number": True, + }, + ), + ( + 0, + 0, + { + "name": "vat", + "parser": "regex", + "regex": r"VAT\s+(\S+)", + "type": "char", + # extract_number on a Text field: silently ignored, + # not surfaced as a bogus template key. + "extract_number": True, + }, + ), + ], + } + ) + composed = record._compose_template_dict() + self.assertTrue(composed["fields"]["amount"]["extract_number"]) + self.assertNotIn("extract_number", composed["fields"]["vat"]) + + # === Filtering by template_type === + + def test_get_templates_filters_by_type_and_active(self): + active = self.Template.create( + { + "name": "acme.purchase.active", + "keywords": "ACME", + "template_type": "purchase_invoice", + } + ) + self.Template.create( + { + "name": "acme.purchase.inactive", + "keywords": "ACME", + "template_type": "purchase_invoice", + "active": False, + } + ) + names = [ + tpl["template_name"] + for tpl in self.Template.get_templates("purchase_invoice") + ] + self.assertIn(active.name, names) + self.assertNotIn("acme.purchase.inactive", names) + + # === Wizard merge === + + def test_wizard_collect_includes_db_templates(self): + self.Template.create( + { + "name": "acme.purchase.wizardtest", + "keywords": "ACME", + "template_type": "purchase_invoice", + } + ) + Wizard = self.env["account.invoice.import"] + with mock.patch( + "odoo.addons.account_invoice_import_invoice2data_db_templates" + ".wizard.account_invoice_import.read_templates", + return_value=[], + ): + collected = Wizard._invoice2data_collect_templates() + names = [tpl["template_name"] for tpl in collected if "template_name" in tpl] + self.assertIn("acme.purchase.wizardtest", names) diff --git a/account_invoice_import_invoice2data_db_templates/views/invoice2data_template.xml b/account_invoice_import_invoice2data_db_templates/views/invoice2data_template.xml new file mode 100644 index 0000000000..6b76783324 --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/views/invoice2data_template.xml @@ -0,0 +1,241 @@ + + + + invoice2data.template.tree + invoice2data.template + + + + + + + + + + + + invoice2data.template.search + invoice2data.template + + + + + + + + + + + + + + invoice2data.template.form + invoice2data.template + +
+
+
+ + +
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Click Refresh candidates in the header to (re)populate this list from the attached PDF. It shows every typed candidate and label match invoice2data's authoring API found — helpful for understanding why a field was proposed (or not) by Suggest fields.
+ +
+ + + + + + + + +
+
+
+ + + +
+
+
+
+ + + Invoice2data Templates + invoice2data.template + tree,form + + +

Author a DB template.

+

+ DB-stored invoice2data templates live alongside the disk-loaded + ones. Attach a sample PDF to the chatter and use the + Suggest fields / Test buttons to iterate. +

+
+
+ + +
diff --git a/account_invoice_import_invoice2data_db_templates/wizard/__init__.py b/account_invoice_import_invoice2data_db_templates/wizard/__init__.py new file mode 100644 index 0000000000..5ffbc4a5dc --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/wizard/__init__.py @@ -0,0 +1,7 @@ +from . import ( + account_invoice_import, + guided_suggest, + import_disk_templates, + pdf_click_suggest, + preview_extraction, +) diff --git a/account_invoice_import_invoice2data_db_templates/wizard/account_invoice_import.py b/account_invoice_import_invoice2data_db_templates/wizard/account_invoice_import.py new file mode 100644 index 0000000000..c7fd67c15d --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/wizard/account_invoice_import.py @@ -0,0 +1,100 @@ +# Copyright 2025-2026 bosd +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). +"""Merge DB-stored invoice2data templates into the import wizard. + +Extends the wizard contributed by ``account_invoice_import_invoice2data`` +to append the active DB templates to the list passed into +``invoice2data.extract_data``. Disk templates remain authoritative; DB +templates are merged in alongside them, so a partner can ship a base set +on disk and override per-deployment in the UI. +""" + +import logging +import os +import shutil +from tempfile import NamedTemporaryFile + +from odoo import _, api, models, tools +from odoo.exceptions import UserError + +logger = logging.getLogger(__name__) + +try: + from invoice2data import extract_data + from invoice2data.extract.loader import read_templates +except ImportError: # pragma: no cover + extract_data = None + read_templates = None + + +class AccountInvoiceImport(models.TransientModel): + _inherit = "account.invoice.import" + + @api.model + def invoice2data_parse_invoice(self, file_data, company): + """Reimplement the wizard's hook to merge DB templates. + + Mirrors the upstream method but replaces the explicit disk-only + template build with disk + DB (via ``_invoice2data_collect_templates``). + Keeps the rest of the flow (OCR fallback via tesseract, then + ``invoice2data_to_parsed_inv``) untouched. + """ + if extract_data is None or read_templates is None: + # Lib missing: propagate the upstream error path. + return super().invoice2data_parse_invoice(file_data, company) + + logger.info("Trying to analyze PDF invoice with invoice2data lib + DB") + with NamedTemporaryFile( + "wb", prefix="odoo-aii-inv2data-pdf-", suffix=".pdf" + ) as fileobj: + fileobj.write(file_data) + fileobj.flush() + templates = self._invoice2data_collect_templates() + try: + result = extract_data(fileobj.name, templates=templates) + except Exception as exc: # noqa: BLE001 + raise UserError( + _("PDF Invoice parsing failed. Error message: %s") % exc + ) from exc + if not result: + result = self._invoice2data_try_tesseract(fileobj.name, templates) + if not result: + return False + return self.invoice2data_to_parsed_inv(result) + + # === Hooks === + + @api.model + def _invoice2data_collect_templates(self): + """Build the (disk + DB) template list passed to ``extract_data``. + + Override this hook in a downstream module to drop the disk set, + filter templates, or stamp in environment-specific ones. + """ + templates = [] + local_dir = tools.config.get("invoice2data_templates_dir", False) + if local_dir and os.path.isdir(local_dir): + templates += read_templates(local_dir) + exclude_built_in = tools.config.get( + "invoice2data_exclude_built_in_templates", False + ) + if not exclude_built_in: + templates += read_templates() + templates += self.env["invoice2data.template"].get_templates("purchase_invoice") + return templates + + def _invoice2data_try_tesseract(self, path, templates): + """OCR fallback path, mirroring the upstream wizard's behaviour.""" + if not shutil.which("tesseract"): + logger.warning("Tesseract fallback unavailable; install tesseract-ocr") + return False + logger.info("Falling back on Tesseract OCR") + try: + from invoice2data.input import tesseract + except ImportError: + return False + try: + return extract_data(path, templates=templates, input_module=tesseract) + except Exception: # noqa: BLE001 + logger.exception("Tesseract fallback failed for %s", path) + return False diff --git a/account_invoice_import_invoice2data_db_templates/wizard/guided_suggest.py b/account_invoice_import_invoice2data_db_templates/wizard/guided_suggest.py new file mode 100644 index 0000000000..3728716ec4 --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/wizard/guided_suggest.py @@ -0,0 +1,213 @@ +# Copyright 2026 bosd +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). +"""Guided-suggest wizard — per-field Keep / Edit / Skip decisions. + +Port of the CLI's copier-style ``_interactive_template`` flow into an +Odoo modal wizard. Instead of the CLI's one-field-at-a-time prompt, the +Odoo variant shows every drafted field in a single editable tree so a +non-technical user sees the whole draft at once, ticks Keep on rows that +look right, edits the regex on rows that need tweaking, and Skips rows +that don't belong — then Apply commits only the kept/edited rows to the +template's ``field_ids``. + +Rationale: the CLI's one-at-a-time UX helps a developer iterate on a +single field. The Odoo persona (non-regex end user) benefits more from +seeing "these five fields were suggested; which are right?" at a glance, +mirroring how the invoice preview shows the whole extraction on one form. +""" + +import logging + +from odoo import _, api, fields, models +from odoo.exceptions import UserError + +_logger = logging.getLogger(__name__) + + +class GuidedSuggestWizard(models.TransientModel): + _name = "invoice2data.template.field.walk.wizard" + _description = "Guided suggest — per-field Keep / Edit / Skip for a template draft" + + template_id = fields.Many2one( + "invoice2data.template", + required=True, + readonly=True, + ondelete="cascade", + ) + template_name = fields.Char(related="template_id.name", readonly=True) + line_ids = fields.One2many( + "invoice2data.template.field.walk.line", + "wizard_id", + ) + source = fields.Selection( + [ + ("deterministic", "Deterministic (find_candidates + labels)"), + ("ai", "AI (generate_template)"), + ], + default="deterministic", + required=True, + help=( + "Which authoring API drafts the field proposals. Deterministic " + "uses the same helpers as the plain 'Suggest fields' button. " + "AI requires the [ai] extra + a configured provider." + ), + ) + + @api.model + def _from_template(self, template, source="deterministic"): + """Build a wizard pre-populated with a fresh draft for ``template``.""" + attachment = template._latest_attachment() + if not attachment: + raise UserError( + _( + "Attach a sample PDF to the chatter before running " + "Guided suggest." + ) + ) + text = template._extract_text(attachment) + try: + if source == "ai": + from invoice2data.ai.template_generator import generate_template + + draft = generate_template(text) + else: + from invoice2data.extract.template_builder import suggested_template + + draft = suggested_template(text) + except ImportError as exc: + hint = ( + _( + "AI draft requires `pip install 'invoice2data[ai]'` + " + "provider env vars. Error: %s" + ) + if source == "ai" + else _("invoice2data >= 1.0 required: %s") + ) + raise UserError(hint % exc) from exc + existing = {row.name for row in template.field_ids} + wizard = self.create({"template_id": template.id, "source": source}) + vals = [] + for i, (fname, spec) in enumerate((draft.get("fields") or {}).items()): + regex, captured = _spec_to_regex_and_preview(spec, text) + vals.append( + ( + 0, + 0, + { + "sequence": i * 10, + "field_name": fname, + "proposed_regex": regex, + "captured_value": captured or "", + "already_in_template": fname in existing, + # Skip-by-default rows the template already has; + # keep-by-default the fresh ones. + "decision": "skip" if fname in existing else "keep", + }, + ) + ) + wizard.line_ids = vals + return wizard + + def action_apply(self): + """Apply the wizard's kept / edited rows to the template's field grid.""" + self.ensure_one() + template = self.template_id + existing = {row.name: row for row in template.field_ids} + added = replaced = skipped = 0 + for line in self.line_ids: + if line.decision == "skip": + skipped += 1 + continue + regex = line.proposed_regex or "" + if not regex: + # A kept row with an empty regex would produce an unusable + # template field; treat as skip and warn in the log. + _logger.info( + "guided_suggest: skipping %r (kept but empty regex)", + line.field_name, + ) + skipped += 1 + continue + vals = {"name": line.field_name, "parser": "regex", "regex": regex} + if line.field_name in existing: + existing[line.field_name].write(vals) + replaced += 1 + else: + template.field_ids = [(0, 0, vals)] + added += 1 + template.last_test_warnings = _( + "Guided suggest applied: %(added)d added, %(replaced)d replaced, " + "%(skipped)d skipped (from %(source)s). Click Test to validate." + ) % { + "added": added, + "replaced": replaced, + "skipped": skipped, + "source": self.source, + } + return {"type": "ir.actions.act_window_close"} + + +class GuidedSuggestLine(models.TransientModel): + _name = "invoice2data.template.field.walk.line" + _description = "Guided suggest line — one proposed field" + _order = "sequence, id" + + wizard_id = fields.Many2one( + "invoice2data.template.field.walk.wizard", + required=True, + ondelete="cascade", + ) + sequence = fields.Integer(default=10) + field_name = fields.Char(readonly=True) + captured_value = fields.Char( + readonly=True, + help=( + "What the proposed regex captures on the sample text. Empty " + "means the regex did not match the sample — either edit it or " + "skip this row." + ), + ) + proposed_regex = fields.Char( + help=( + "The regex that will be stored on the template field. Editable — " + "if you tweak it, keep the row selected as 'Keep' and Apply." + ), + ) + already_in_template = fields.Boolean( + readonly=True, + help=( + "True when the template already has a field with this name. " + "Applying with decision=Keep or Edit will REPLACE the existing " + "regex." + ), + ) + decision = fields.Selection( + [("keep", "Keep"), ("edit", "Edit"), ("skip", "Skip")], + default="keep", + required=True, + help=( + "Keep = apply the proposed regex as-is. Edit = same as Keep " + "(so the tweaked regex is used). Skip = don't touch the " + "template for this field." + ), + ) + + +def _spec_to_regex_and_preview(spec, text): + """Return ``(regex_str, captured_or_none)`` for a template_builder spec. + + ``spec`` may be a bare regex string or a ``{regex, replace}`` dict + (both shapes are what `suggested_template` returns). Uses + ``preview_field`` from the same module to compute the captured value + on ``text``. + """ + try: + from invoice2data.extract.template_builder import field_regex, preview_field + + regex = field_regex(spec) + captured = preview_field(spec, text) + except Exception as exc: # noqa: BLE001 -- diagnostic-only + _logger.debug("guided_suggest: preview failed: %s", exc) + regex = spec if isinstance(spec, str) else spec.get("regex", "") + captured = None + return regex, captured diff --git a/account_invoice_import_invoice2data_db_templates/wizard/guided_suggest.xml b/account_invoice_import_invoice2data_db_templates/wizard/guided_suggest.xml new file mode 100644 index 0000000000..3f80a1cebc --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/wizard/guided_suggest.xml @@ -0,0 +1,51 @@ + + + + invoice2data.template.field.walk.wizard.form + invoice2data.template.field.walk.wizard + +
+ +
+

Guided suggest for

+
Review each proposed field. Keep accepts the row as-is; Edit means you tweaked the regex and want that version applied; Skip leaves the template untouched for that field. Rows the template already has are pre-selected to Skip so you don't accidentally overwrite manual work.
+
+ + + + + + + + + + + + + +
+
+
+
+
+
+
diff --git a/account_invoice_import_invoice2data_db_templates/wizard/import_disk_templates.py b/account_invoice_import_invoice2data_db_templates/wizard/import_disk_templates.py new file mode 100644 index 0000000000..00161d887d --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/wizard/import_disk_templates.py @@ -0,0 +1,193 @@ +# Copyright 2026 bosd +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). +"""Wizard to import bundled/disk-loaded invoice2data templates into the DB. + +Copies YAML/JSON templates shipped by the invoice2data library (or by the +sysadmin's ``INVOICE2DATA_TEMPLATES_DIR``) into ``invoice2data.template`` +records, giving the author a starting point they can then customise via +the field editor / JSON tab / keywords tags. + +The disk template dict is serialised into the record's ``template`` JSON +field verbatim (so nothing about the source template is lost — including +options, lines blocks, and any keys the field editor does not surface). +Keywords / exclude_keywords are ALSO populated as m2m tags so they show +up in the header of the imported record. +""" + +import json +import logging +import re + +from odoo import _, api, fields, models +from odoo.exceptions import UserError + +_logger = logging.getLogger(__name__) + + +class Invoice2dataTemplateImportDiskWizard(models.TransientModel): + _name = "invoice2data.template.import.disk.wizard" + _description = "Import bundled invoice2data templates into the DB" + + name_regex = fields.Char( + string="Filter template name (regex)", + help=( + "Only templates whose filename matches this Python regex are " + "imported. Leave blank to import ALL bundled templates " + "(currently ~215). Example: '^nl\\.' to import only Dutch " + "vendors, 'shell' to import both nl.shell.* and be.shell.*." + ), + ) + mode = fields.Selection( + selection=[ + ("skip_existing", "Skip if name already exists"), + ("overwrite", "Overwrite existing records"), + ], + default="skip_existing", + required=True, + ) + dry_run = fields.Boolean( + default=False, + help=( + "Report how many templates WOULD be imported without creating or " + "modifying any records." + ), + ) + result_summary = fields.Text(readonly=True) + + def action_import(self): + """Read disk templates, filter, create/update DB records. + + Returns a client action re-opening the wizard so the user can see + ``result_summary`` populated with counts. + """ + self.ensure_one() + try: + from invoice2data.extract.loader import read_templates + except ImportError as exc: + raise UserError( + _("invoice2data is not importable on the server: %s") % exc + ) from exc + + try: + pattern = re.compile(self.name_regex) if self.name_regex else None + except re.error as exc: + raise UserError( + _("Invalid regex %(rx)r: %(err)s") % {"rx": self.name_regex, "err": exc} + ) from exc + + disk_templates = read_templates() + Template = self.env["invoice2data.template"] + Keyword = self.env["invoice2data.template.keyword"] + keyword_cache = {} + + considered = created = updated = skipped = 0 + errors = [] + + for disk_tpl in disk_templates: + name = disk_tpl.get("template_name", "") + if pattern and not pattern.search(name): + continue + considered += 1 + existing = Template.search([("name", "=", name)], limit=1) + if existing and self.mode == "skip_existing": + skipped += 1 + continue + if self.dry_run: + if existing: + updated += 1 + else: + created += 1 + continue + + try: + vals = self._vals_from_disk_template(disk_tpl, Keyword, keyword_cache) + except Exception as exc: # noqa: BLE001 -- report per-template + errors.append("%s: %s" % (name, exc)) + _logger.warning( + "import_disk_templates: failed to import %r: %s", name, exc + ) + continue + + if existing: + existing.write(vals) + updated += 1 + else: + Template.create(vals) + created += 1 + + summary_lines = [ + _("Considered: %d") % considered, + _("Created: %d") % created, + _("Updated: %d") % updated, + _("Skipped: %d (already existed)") % skipped, + ] + if errors: + summary_lines.append("") + summary_lines.append(_("Errors (%d):") % len(errors)) + summary_lines.extend(errors[:20]) + if len(errors) > 20: + summary_lines.append( + _("... %d more suppressed; check server log") % (len(errors) - 20) + ) + if self.dry_run: + summary_lines.insert(0, _("[DRY RUN — no records touched]")) + self.result_summary = "\n".join(summary_lines) + return { + "type": "ir.actions.act_window", + "res_model": self._name, + "res_id": self.id, + "view_mode": "form", + "target": "new", + } + + @staticmethod + def _clean_template_dict(disk_tpl): + """Strip runtime-only keys before serialising the disk template.""" + out = dict(disk_tpl) # InvoiceTemplate is a dict subclass + # `template_name` lives on the DB record's `name` field; strip it + # from the JSON blob so a rename works. + out.pop("template_name", None) + return out + + @api.model + def _vals_from_disk_template(self, disk_tpl, Keyword, keyword_cache): + """Turn one on-disk InvoiceTemplate into create/write values.""" + name = disk_tpl.get("template_name", "") + if not name: + raise ValueError("template has no template_name; cannot import") + keyword_ids = self._ensure_keyword_ids( + disk_tpl.get("keywords") or [], Keyword, keyword_cache + ) + exclude_keyword_ids = self._ensure_keyword_ids( + disk_tpl.get("exclude_keywords") or [], Keyword, keyword_cache + ) + return { + "name": name, + "priority": int(disk_tpl.get("priority") or 5), + "keywords": [(6, 0, keyword_ids)], + "exclude_keywords": [(6, 0, exclude_keyword_ids)], + "template": json.dumps( + self._clean_template_dict(disk_tpl), indent=2, default=str + ), + "active": True, + } + + @staticmethod + def _ensure_keyword_ids(names, Keyword, cache): + """Return ids of keyword records for each name, creating on miss.""" + ids = [] + for raw in names: + name = raw if isinstance(raw, str) else str(raw) + name = name.strip() + if not name: + continue + if name in cache: + ids.append(cache[name]) + continue + existing = Keyword.search([("name", "=", name)], limit=1) + if existing: + cache[name] = existing.id + else: + cache[name] = Keyword.create({"name": name}).id + ids.append(cache[name]) + return ids diff --git a/account_invoice_import_invoice2data_db_templates/wizard/import_disk_templates.xml b/account_invoice_import_invoice2data_db_templates/wizard/import_disk_templates.xml new file mode 100644 index 0000000000..3215f70e46 --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/wizard/import_disk_templates.xml @@ -0,0 +1,49 @@ + + + + invoice2data.template.import.disk.wizard.form + invoice2data.template.import.disk.wizard + +
+ + + + + + + + +
+
+
+
+
+ + + Import disk templates + invoice2data.template.import.disk.wizard + form + new + + + +
diff --git a/account_invoice_import_invoice2data_db_templates/wizard/pdf_click_suggest.py b/account_invoice_import_invoice2data_db_templates/wizard/pdf_click_suggest.py new file mode 100644 index 0000000000..955cfd50ef --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/wizard/pdf_click_suggest.py @@ -0,0 +1,201 @@ +# Copyright 2026 bosd +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). +"""Click-to-suggest wizard — coord-entry harness for the first slice. + +This is the SECOND half of the design brief's click-to-suggest design +(``the internal design brief). The first half — +the server-side pdfium resolution in ``models/pdf_click.py`` — is +feature-complete: given a ``(page, x, y)`` or ``(page, x, y, w, h)``, it +returns a field proposal, its captured value, and a pdftotext cross- +check status. + +The remaining piece is the front-end: a wizard that renders the sample +PDF as a page image with a clickable / draggable overlay so the author +doesn't have to type coordinates. That's an OWL widget and lands in a +follow-up PR (its design is stable — see the design brief). + +Until the OWL widget arrives, this wizard exposes the server pipeline +as a *coord-entry harness*: enter page + x + y (or drag rect), click +Preview, see the proposal + captured value + cross-check badge. Useful +for validating the pipeline end-to-end today and for regressing it in +tests; the OWL widget will replace only the coord-entry inputs, not +the resolver or the ``Apply`` flow underneath. +""" + +import base64 +import logging +import tempfile + +from odoo import _, api, fields, models +from odoo.exceptions import UserError + +_logger = logging.getLogger(__name__) + + +class PdfClickSuggestWizard(models.TransientModel): + _name = "invoice2data.template.pdf.click.wizard" + _description = "Click-to-suggest — pipeline harness (OWL widget in a follow-up)" + + template_id = fields.Many2one( + "invoice2data.template", required=True, readonly=True, ondelete="cascade" + ) + template_name = fields.Char(related="template_id.name", readonly=True) + + page = fields.Integer(default=0, help="0-based page index on the sample PDF.") + x_pt = fields.Float(string="x (pt)", help="Top-left origin, 72 dpi points.") + y_pt = fields.Float(string="y (pt)", help="Top-left origin, 72 dpi points.") + w_pt = fields.Float( + string="width (pt)", + help="0 = click. Non-zero + non-zero height = drag rectangle → area field.", + ) + h_pt = fields.Float(string="height (pt)") + + # Populated by Preview. + proposal_field = fields.Char(readonly=True) + proposal_value = fields.Char(readonly=True) + proposal_line = fields.Text(readonly=True) + proposal_spec = fields.Text( + readonly=True, help="The regex or {regex, replace} dict to store." + ) + proposal_note = fields.Text(readonly=True) + pdftotext_status = fields.Selection( + [ + ("same", "pdftotext: same value"), + ("no-match", "pdftotext: no match (backend drift)"), + ("unavailable", "pdftotext: unavailable"), + ], + readonly=True, + ) + field_key_id = fields.Many2one( + "invoice2data.field.key", + string="Canonical field", + help="Adjust here if the proposal's field is wrong before Apply.", + ) + + def action_preview(self): + """Resolve the current (page, x, y[, w, h]) to a proposal.""" + self.ensure_one() + from ..models import pdf_click + + attachment = self.template_id._latest_attachment() + if not attachment: + raise UserError(_("Attach a sample PDF to the template's chatter first.")) + with tempfile.NamedTemporaryFile( + "wb", prefix="i2d-click-", suffix=".pdf", delete=False + ) as fh: + fh.write(base64.b64decode(attachment.datas)) + path = fh.name + try: + if self.w_pt and self.h_pt: + result = pdf_click.suggest_area( + path, self.page, (self.x_pt, self.y_pt, self.w_pt, self.h_pt) + ) + else: + result = pdf_click.suggest_at(path, self.page, self.x_pt, self.y_pt) + except Exception as exc: # noqa: BLE001 -- surface via the form + self.proposal_note = _("Resolver error: %s") % exc + return self._reopen() + if "error" in result: + self.proposal_note = _("No proposal: %s") % result["error"] + self.proposal_field = "" + self.proposal_value = "" + self.proposal_spec = "" + return self._reopen() + spec = result.get("spec") + self.proposal_field = result.get("field") or "" + self.proposal_value = result.get("value") or "" + self.proposal_line = result.get("line") or "" + self.proposal_spec = spec if isinstance(spec, str) else _json_dumps(spec) + self.proposal_note = result.get("note") or "" + self.pdftotext_status = result.get("pdftotext_status") or "unavailable" + # Pre-select the canonical field key if we have one. + if self.proposal_field: + key = self.env["invoice2data.field.key"].search( + [("name", "=", self.proposal_field)], limit=1 + ) + if key: + self.field_key_id = key.id + return self._reopen() + + def action_apply(self): + """Append the previewed proposal as a new field row on the template. + + Also writes ``input_module='pdfium'`` on the template + (cross-backend-drift guard, per design brief §Q2 recommendation (a)), + and stores the click hint + sample_value on the new field row so + Test can regress the capture and the future OWL widget can + reopen the click position. + """ + self.ensure_one() + if not self.proposal_field or not self.proposal_spec: + raise UserError(_("Nothing to apply — click Preview first.")) + template = self.template_id + # Pin the backend if not already set. Never OVERWRITE an + # explicit non-pdfium choice — the author may have set it on + # purpose. + if not template.input_module: + template.input_module = "pdfium" + # Prefer the canonical-field selection if the user overrode it. + field_name = ( + self.field_key_id.name if self.field_key_id else self.proposal_field + ) + # Unpack spec (JSON dict) into regex + replace, or use as-is if bare. + raw = self.proposal_spec.strip() + if raw.startswith("{"): + try: + spec = _json_loads(raw) + except Exception: # noqa: BLE001 + spec = {"regex": raw} + else: + spec = {"regex": raw} + vals = { + "template_id": template.id, + "name": field_name, + "parser": spec.get("parser", "regex"), + "regex": spec.get("regex", ""), + "hint_page": self.page, + "hint_x": self.x_pt, + "hint_y": self.y_pt, + "hint_w": self.w_pt, + "hint_h": self.h_pt, + "sample_line": (self.proposal_line or "")[:255], + "sample_value": (self.proposal_value or "")[:255], + } + if "replace" in spec and isinstance(spec["replace"], (list, tuple)): + if len(spec["replace"]) >= 2: + vals["replace_pattern"] = spec["replace"][0] + vals["replace_repl"] = spec["replace"][1] + # If the field already exists on the template, replace it (matches + # the guided-suggest wizard's Keep-replaces-existing behaviour). + existing = template.field_ids.filtered(lambda r: r.name == field_name) + if existing: + existing[:1].write({k: v for k, v in vals.items() if k != "template_id"}) + else: + self.env["invoice2data.template.field"].create(vals) + return {"type": "ir.actions.act_window_close"} + + def _reopen(self): + return { + "type": "ir.actions.act_window", + "res_model": self._name, + "res_id": self.id, + "view_mode": "form", + "target": "new", + } + + @api.model + def _from_template(self, template): + wizard = self.create({"template_id": template.id}) + return wizard + + +def _json_dumps(obj): + import json + + return json.dumps(obj, indent=2, default=str) + + +def _json_loads(text): + import json + + return json.loads(text) diff --git a/account_invoice_import_invoice2data_db_templates/wizard/pdf_click_suggest.xml b/account_invoice_import_invoice2data_db_templates/wizard/pdf_click_suggest.xml new file mode 100644 index 0000000000..f7e2dd4b5f --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/wizard/pdf_click_suggest.xml @@ -0,0 +1,70 @@ + + + + invoice2data.template.pdf.click.wizard.form + invoice2data.template.pdf.click.wizard + +
+ +
+

Click to suggest —

+
First-slice harness. Enter a page and (x, y) in points (72 dpi, top-left origin). A non-zero width and height turn a click into a drag rectangle → area field. Click Preview to see what the resolver proposes. The clickable PDF viewer lives in a follow-up PR.
+
+ + + + + + + + + + + + + + + + + + + + +
+
+
+
+
+
+
diff --git a/account_invoice_import_invoice2data_db_templates/wizard/preview_extraction.py b/account_invoice_import_invoice2data_db_templates/wizard/preview_extraction.py new file mode 100644 index 0000000000..d6e6efec1a --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/wizard/preview_extraction.py @@ -0,0 +1,310 @@ +# Copyright 2026 bosd +# License AGPL-3.0 or later (http://www.gnu.org/licenses/agpl). +"""Preview an invoice2data extraction as it would look on the target Odoo model. + +Renders the fields invoice2data would fill on the target document +(currently account.move for purchase_invoice templates) in an +invoice-shaped read-only form: partner block + invoice header + line +items + totals footer. Non-technical template authors read this instead +of the raw JSON blob to answer "does the extraction look right?". + +The wizard is a TransientModel — no records persist. The Test / Test +isolated buttons run extract_data as before (populating the template +record's last_test_result / last_test_warnings for the diagnostics +banner) and return an act_window that opens this wizard in a modal. +""" + +import logging +from datetime import date, datetime + +from odoo import api, fields, models + +_logger = logging.getLogger(__name__) + + +class Invoice2dataTemplatePreview(models.TransientModel): + _name = "invoice2data.template.preview" + _description = "Invoice2data extraction preview" + + template_id = fields.Many2one( + "invoice2data.template", + required=True, + readonly=True, + ondelete="cascade", + ) + template_name = fields.Char(related="template_id.name", readonly=True) + winning_template_name = fields.Char( + readonly=True, + help=( + "The template invoice2data actually picked — may differ from " + "the one being authored when a sibling / disk template wins." + ), + ) + mode = fields.Selection( + [("isolated", "Isolated"), ("db_only", "DB only"), ("full", "Disk + DB")], + readonly=True, + ) + + # --- Partner block (mirrors res.partner shape) -------------------- + partner_name = fields.Char(readonly=True) + partner_vat = fields.Char(string="VAT", readonly=True) + partner_email = fields.Char(readonly=True) + partner_street = fields.Char(readonly=True) + partner_zip = fields.Char(string="ZIP", readonly=True) + partner_city = fields.Char(readonly=True) + partner_country_code = fields.Char(string="Country", readonly=True) + partner_id = fields.Many2one( + "res.partner", + string="Matched partner", + readonly=True, + help=( + "Best partner match found in Odoo. Tries VAT first, then a " + "case-insensitive name search. Empty means the import would " + "either create a new partner or reject the invoice depending " + "on the import wizard configuration." + ), + ) + + # --- Invoice header (mirrors account.move shape) ------------------ + invoice_ref = fields.Char(string="Vendor invoice / reference", readonly=True) + invoice_date = fields.Date(readonly=True) + invoice_date_due = fields.Date(string="Due date", readonly=True) + payment_reference = fields.Char(readonly=True) + currency_id = fields.Many2one("res.currency", readonly=True) + + # --- Amount footer ----------------------------------------------- + amount_untaxed = fields.Monetary(currency_field="currency_id", readonly=True) + amount_tax = fields.Monetary(currency_field="currency_id", readonly=True) + amount_total = fields.Monetary(currency_field="currency_id", readonly=True) + + # --- Lines -------------------------------------------------------- + line_ids = fields.One2many( + "invoice2data.template.preview.line", + "preview_id", + readonly=True, + ) + tax_line_ids = fields.One2many( + "invoice2data.template.preview.tax", "preview_id", readonly=True + ) + + # --- Diagnostics -------------------------------------------------- + missing_fields = fields.Char( + readonly=True, + help=( + "Canonical field names that the invoice-import wizard would " + "typically want but this extraction did not yield. Empty = " + "nothing missing." + ), + ) + warnings = fields.Text(readonly=True) + raw_json = fields.Text(readonly=True) + + # --- Constructor --------------------------------------------------- + + @api.model + def _from_extraction(self, template, extracted, mode, warnings): + """Build a preview record from an invoice2data extraction dict. + + Args: + template (invoice2data.template): the template being tested. + extracted (dict): what extract_data returned (may be empty). + mode (str): 'isolated' / 'db_only' / 'full'. + warnings (list[str]): the diagnostic lines from _run_test. + """ + vals = { + "template_id": template.id, + "mode": mode, + "warnings": "\n".join(warnings) if warnings else "", + "raw_json": _dumps(extracted) if extracted else "", + } + if extracted: + vals["winning_template_name"] = extracted.get("template_name") or "" + vals.update(self._extract_partner_vals(self.env, extracted)) + vals.update(self._extract_header_vals(self.env, extracted)) + vals.update(self._extract_amount_vals(extracted)) + vals["missing_fields"] = ", ".join(self._missing(extracted)) or "" + preview = self.create(vals) + preview.line_ids = [ + (0, 0, spec) for spec in self._line_vals(extracted.get("lines") or []) + ] + preview.tax_line_ids = [ + (0, 0, spec) + for spec in self._tax_line_vals(extracted.get("tax_lines") or []) + ] + return preview + + # --- Field extractors -------------------------------------------- + + @staticmethod + def _extract_partner_vals(env, extracted): + vals = { + "partner_name": extracted.get("partner_name") or extracted.get("issuer"), + "partner_vat": extracted.get("vat"), + "partner_email": extracted.get("partner_email"), + "partner_street": extracted.get("partner_street"), + "partner_zip": extracted.get("partner_zip"), + "partner_city": extracted.get("partner_city"), + "partner_country_code": extracted.get("country_code"), + } + # Best-effort partner match — VAT first (exact, case-insensitive + # after stripping spaces), name fallback (case-insensitive + # substring). Skips silently on any lookup error so a broken + # search filter can't kill the preview. + Partner = env["res.partner"].sudo() + try: + partner = env["res.partner"].browse() + if vals["partner_vat"]: + clean = vals["partner_vat"].replace(" ", "").upper() + partner = Partner.search([("vat", "=ilike", clean)], limit=1) + if not partner and vals["partner_name"]: + partner = Partner.search( + [("name", "=ilike", vals["partner_name"])], limit=1 + ) + vals["partner_id"] = partner.id if partner else False + except Exception as exc: # noqa: BLE001 -- diagnostic-only + _logger.debug("preview: partner lookup failed: %s", exc) + return vals + + @staticmethod + def _extract_header_vals(env, extracted): + vals = { + "invoice_ref": extracted.get("invoice_number"), + "invoice_date": _to_date(extracted.get("date")), + "invoice_date_due": _to_date(extracted.get("date_due")), + "payment_reference": extracted.get("payment_reference"), + } + code = extracted.get("currency") + if code: + currency = ( + env["res.currency"] + .with_context(active_test=False) + .search([("name", "=", code)], limit=1) + ) + vals["currency_id"] = currency.id if currency else False + return vals + + @staticmethod + def _extract_amount_vals(extracted): + return { + "amount_total": extracted.get("amount") or 0.0, + "amount_untaxed": extracted.get("amount_untaxed") or 0.0, + "amount_tax": extracted.get("amount_tax") or 0.0, + } + + @staticmethod + def _missing(extracted): + wanted = ( + "issuer", + "date", + "amount", + "invoice_number", + "partner_name", + "vat", + "currency", + ) + return [name for name in wanted if not extracted.get(name)] + + @staticmethod + def _line_vals(rows): + for i, row in enumerate(rows or []): + if not isinstance(row, dict): + continue + yield { + "sequence": i, + "name": row.get("name") or row.get("description") or "", + "product": row.get("product") or row.get("code"), + "qty": _to_float(row.get("qty")), + "uom": row.get("uom") or row.get("unece_code") or "", + "price_unit": _to_float(row.get("price_unit")), + "price_subtotal": _to_float( + row.get("price_subtotal") or row.get("price_total") + ), + } + + @staticmethod + def _tax_line_vals(rows): + for i, row in enumerate(rows or []): + if not isinstance(row, dict): + continue + yield { + "sequence": i, + "name": row.get("name") or row.get("tax_id") or "", + "base": _to_float(row.get("base")), + "amount": _to_float(row.get("amount")), + "unece_categ_code": row.get("unece_categ_code") + or row.get("tax_code") + or "", + } + + +class Invoice2dataTemplatePreviewLine(models.TransientModel): + _name = "invoice2data.template.preview.line" + _description = "Invoice2data preview line" + _order = "sequence, id" + + preview_id = fields.Many2one( + "invoice2data.template.preview", required=True, ondelete="cascade" + ) + sequence = fields.Integer(default=10) + # pylint: disable=attribute-string-redundant + name = fields.Char(string="Description", readonly=True) + product = fields.Char(string="Product / code", readonly=True) + qty = fields.Float(string="Qty", readonly=True) + uom = fields.Char(string="UoM", readonly=True) + price_unit = fields.Float(string="Unit price", readonly=True) + price_subtotal = fields.Float(string="Subtotal", readonly=True) + + +class Invoice2dataTemplatePreviewTax(models.TransientModel): + _name = "invoice2data.template.preview.tax" + _description = "Invoice2data preview tax line" + _order = "sequence, id" + + preview_id = fields.Many2one( + "invoice2data.template.preview", required=True, ondelete="cascade" + ) + sequence = fields.Integer(default=10) + name = fields.Char(string="Tax", readonly=True) + unece_categ_code = fields.Char(string="UNECE code", readonly=True) + base = fields.Float(readonly=True) + amount = fields.Float(readonly=True) + + +# --- Coercion helpers ------------------------------------------------- + + +def _to_date(value): + """Coerce whatever invoice2data emitted for a date field to a Date.""" + if isinstance(value, date) and not isinstance(value, datetime): + return value + if isinstance(value, datetime): + return value.date() + if not value: + return False + if isinstance(value, str): + # invoice2data usually normalises to ISO — accept but don't be + # strict, the wizard is preview-only. + for fmt in ("%Y-%m-%d", "%Y-%m-%dT%H:%M:%S", "%d/%m/%Y"): + try: + return datetime.strptime(value[: len(fmt) + 4], fmt).date() + except ValueError: + continue + return False + + +def _to_float(value): + if value in (None, "", False): + return 0.0 + try: + return float(value) + except (TypeError, ValueError): + try: + return float(str(value).replace(",", ".")) + except (TypeError, ValueError): + return 0.0 + + +def _dumps(obj): + import json + + return json.dumps(obj, indent=2, default=str, ensure_ascii=False) diff --git a/account_invoice_import_invoice2data_db_templates/wizard/preview_extraction.xml b/account_invoice_import_invoice2data_db_templates/wizard/preview_extraction.xml new file mode 100644 index 0000000000..1de9d704f2 --- /dev/null +++ b/account_invoice_import_invoice2data_db_templates/wizard/preview_extraction.xml @@ -0,0 +1,122 @@ + + + + invoice2data.template.preview.form + invoice2data.template.preview + +
+ +
+

+ +

+
+ Preview of the invoice that would be created if invoice2data ran on this PDF today. + + Winning template: · Mode: + +
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
invoice2data did not return an extraction — see the Diagnostics tab and the template's Test results tab for why.
+
+
+ + + + + + +
+ +
+
+
+
diff --git a/requirements.txt b/requirements.txt index f39afbaee2..f09a668680 100644 --- a/requirements.txt +++ b/requirements.txt @@ -2,6 +2,7 @@ dateparser dotty-dict factur-x +invoice2data ovh phonenumbers pydifact diff --git a/setup/account_invoice_import_invoice2data/odoo/addons/account_invoice_import_invoice2data b/setup/account_invoice_import_invoice2data/odoo/addons/account_invoice_import_invoice2data new file mode 120000 index 0000000000..630cb905fe --- /dev/null +++ b/setup/account_invoice_import_invoice2data/odoo/addons/account_invoice_import_invoice2data @@ -0,0 +1 @@ +../../../../account_invoice_import_invoice2data \ No newline at end of file diff --git a/setup/account_invoice_import_invoice2data/setup.py b/setup/account_invoice_import_invoice2data/setup.py new file mode 100644 index 0000000000..28c57bb640 --- /dev/null +++ b/setup/account_invoice_import_invoice2data/setup.py @@ -0,0 +1,6 @@ +import setuptools + +setuptools.setup( + setup_requires=['setuptools-odoo'], + odoo_addon=True, +) diff --git a/setup/account_invoice_import_invoice2data_db_templates/odoo/addons/account_invoice_import_invoice2data_db_templates b/setup/account_invoice_import_invoice2data_db_templates/odoo/addons/account_invoice_import_invoice2data_db_templates new file mode 120000 index 0000000000..9edd3dab7d --- /dev/null +++ b/setup/account_invoice_import_invoice2data_db_templates/odoo/addons/account_invoice_import_invoice2data_db_templates @@ -0,0 +1 @@ +../../../../account_invoice_import_invoice2data_db_templates \ No newline at end of file diff --git a/setup/account_invoice_import_invoice2data_db_templates/setup.py b/setup/account_invoice_import_invoice2data_db_templates/setup.py new file mode 100644 index 0000000000..28c57bb640 --- /dev/null +++ b/setup/account_invoice_import_invoice2data_db_templates/setup.py @@ -0,0 +1,6 @@ +import setuptools + +setuptools.setup( + setup_requires=['setuptools-odoo'], + odoo_addon=True, +)