Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

James.yang/lexer refactor #11

Closed
wants to merge 62 commits into from
Closed

James.yang/lexer refactor #11

wants to merge 62 commits into from

Conversation

JamesYang007
Copy link
Owner

First draft of lexer.

@JamesYang007
Remove all classes
40eeee5
@JamesYang007
Include array
e6d4b54
@JamesYang007
Update logic for routine and state
07049e9
@JamesYang007
Update comments
17857af
@JamesYang007
Add core directory and separate out core stuff from parse
686d3e2
@JamesYang007
Add example implementation of read and slash routines
aff6870
@JamesYang007
Add core material to simplify logic
4e8f076
@JamesYang007
Change pointer syntax to C++ style
a1f46aa
@JamesYang007
Implement back the array of parser routine functions
62b6e43
@JamesYang007
Add RoutineArray and a compile-time check to see if RoutineArray has …
2ec6c9c 
…been constructed properly
@JamesYang007
Use smart pointer to fclose FILE object
2fe54a0
@JamesYang007
Change shared ptr to unique ptr
973aa99
@JamesYang007
Add test directory
db78c55
@JamesYang007
Modify indenting in exceptions
3c4cbe0
@JamesYang007
Add CMake for tests
e3fa395
@JamesYang007
Add unittest for symbol
81569fa
@JamesYang007
Rename parse_unittest to parse_unittests
40415dc
@JamesYang007
Migrate to C++14
144d627
@JamesYang007
Add automated routine array generation
6fdda04
@JamesYang007
Remove commented pseudocode in parse_dir
af311b1
@JamesYang007
Add slash unit test
9da546d
@JamesYang007
Add assert in slash routine
2d6c4f9
@JamesYang007
Link with nlohmann_json
13a5fae
@JamesYang007
Add utility
9d7a2e4
@JamesYang007
Remove parse_dir and move file_deleter
d0f015c
@JamesYang007
Add assert in read routine
f386211
@JamesYang007
Replace while-loop with for-loop and use switch-case
685051b
@JamesYang007
Rename routine unittest specific to read and created separate fixture
b61120e
@JamesYang007
Update cmakelists and finish unittesting with minor impl changes to r…
6b23e48 
…ead routine
@JamesYang007
Change test exec names
8b8b754
@JamesYang007 JamesYang007 changed the base branch from master to james.yang/lexer_parser January 13, 2020 06:37
@JamesYang007 JamesYang007 changed the base branch from james.yang/lexer_parser to james.yang/cmake_init January 13, 2020 06:37
@JamesYang007 JamesYang007 force-pushed the james.yang/lexer_refactor branch from 50de6bc to 150bb43 Compare January 13, 2020 06:43
JamesYang007
JamesYang007 commented Jan 13, 2020
View reviewed changes
Copy link
Owner Author

@JamesYang007 JamesYang007 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed

src/core/status.hpp Outdated Show resolved Hide resolved
kentjhall
kentjhall requested changes Jan 13, 2020
View reviewed changes
Copy link
Collaborator

@kentjhall kentjhall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks really awesome! The big changes are to change to processing one token at a time (the process function should just return a Token), and to change the file_reader to take a FILE * and not worry about opening/closing.

src/core/status.hpp Outdated Show resolved Hide resolved
src/io/file_reader.hpp Outdated Show resolved Hide resolved
src/core/lexer.hpp
status_.tokens.reserve(DEFAULT_TOKEN_ARR_SIZE);
}

void process()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's change the lexer to process one token at a time!

src/core/status.hpp Show resolved Hide resolved
kentjhall
kentjhall requested changes Jan 15, 2020
View reviewed changes
Copy link
Collaborator

@kentjhall kentjhall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just the things we've talked about

src/core/lexer_routines.hpp
@@ -0,0 +1,300 @@
#pragma once
#include "token.hpp"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make these includes relative paths from src/

src/core/symbol.hpp
BEGIN_BLOCK_COMMENT,
END_BLOCK_COMMENT,
// special tags
TAGNAME,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need symbols for each tag individually mb :'(

@JamesYang007
Copy link
Owner Author

Closing this PR for now since we're moving towards Trie implementation. See #13

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kentjhall kentjhall kentjhall requested changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@JamesYang007 @kentjhall