Operator info from Operators.YML file (in MathicsScanner) #1180
Conversation
New package "stopit" seems to let things run much longer on some OS's and Python platforms in certain cases
Also, move OPERATOR_DATA to mathics.core.parser.operators and pick up ROOT_DIR from mathics.settings
rocky
force-pushed
the
operator-info-from-JSON
branch
from
161f3b8 to
8b3f7b1
Compare
|
Since we test all new Builtin functions without pre-defined meaning including the operator form of that, these have to exist in the scanner. Some of these unicode operator symbols are currently missing. I will be making another round in MathicsScanner to pick up all of these such operators from the YML soon. |
|
What about using a blacklist in mathics-corr, to avoid loading the incomplete symbols? |
I'd rather do a relatively small amount of work to finish that aspect than add some code that will be ripped out soon. There will be many other things that could be improved that can wait and be done. |
|
Ok,when I get some time I'll do it. |
and put initialization code in `mathics.builtin.no_meaning` inside an initialization function.
We have a dependency problem in mathics-scanner where code is performed in tokeniser loading. And the script that creates a needed JSON file imports mathics_scanner which imports the tokenser which needs that JSON file to pre-exist. The workaround here is to not import is_symbol_name from mathics_scanner but from mathics_scanner.tokeniser By doing this when the JSON-creating program imports mathics_scanner it does not import the module that requires the JSON file. An attempt was made to delay JSON-reading, but something is still whacky, so that doesn't work yet. Punt on this aspect for later.
pkg_resources is deprecated
…-core into operator-info-from-JSON
Makefile
Outdated
| @@ -142,10 +142,9 @@ latexdoc texdoc doc: | |||
| (cd mathics/doc/latex && $(MAKE) doc) | |||
|
|
|||
| #: Build JSON ASCII to unicode opcode table and operator table | |||
| mathics/data/op-tables.json mathics/data/operators.json: | |||
| mathics/data/operator-tables.json mathics/data/op-tables.json mathics/data/operators.json: | |||
There was a problem hiding this comment.
operators.json is not in mathics-core/mathics/data but in mathics_scanner/data/
There was a problem hiding this comment.
(I should have put a line in mathics_scanner.generate.build_operator_tables that prints where operators.json is stored...)
SYMBOLS_MANIFEST.txt
Outdated
| @@ -134,8 +134,10 @@ System`Automatic | |||
| System`Axes | |||
| System`Axis | |||
| System`Background | |||
| System`Backslash | |||
There was a problem hiding this comment.
@rocky, the changes in the SYMBOLS_MANIFEST.txt are already done...
There was a problem hiding this comment.
Something I ran must have changed this. I definitely have not been changing this by hand.
At any rate, just fix whatever problem you encounter.
There was a problem hiding this comment.
It is fixed automatically once you run the test locally, and the test passes. It does not seem to fail here. What is failing is the localization of the operator.json file, that is not generated in the place where mathics_scanner.tokenizer looks for it.
|
I had considered this a dead branch that should be closed and deleted. At some point this branch was previously merged. Locally I forgot to delete it and committed on top things that I think were also committed to a different branch. |
|
Ok, then we need to adjust the mathics tests in mathics scanner |
This is an extension and follow-on to #1174 .
More of the data encoded in
mathics.core.parsers.operatorsnow comes from JSON. And all of the operators without builtin-meanings are included.There will be some more work to fix up the MANIFESTs (@mmatera are you up for this? And some additional changes would be nice in mathicsscanner to include all of the operators without built-in meanings. Right now, these are not pulled from JSON and are incomplete.