Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize big stack allocation (move to heap) #3016

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

mgreter
Copy link
Contributor

@mgreter mgreter commented Oct 23, 2019

No description provided.

@mgreter mgreter self-assigned this Oct 23, 2019
src/file.cpp Show resolved Hide resolved
@glebm
Copy link
Contributor

glebm commented Oct 23, 2019

Stack allocation is free but heap isn't; how is this an optimization?

@mgreter
Copy link
Contributor Author

mgreter commented Oct 23, 2019

Stack allocation is free but heap isn't; how is this an optimization?

Out of stack issues? https://docs.microsoft.com/en-us/visualstudio/code-quality/c6262?view=vs-2019

// windows unicode filepaths are encoded in utf16
std::string abspath(join_paths(get_cwd(), path));
if (!(abspath[0] == '/' && abspath[1] == '/')) {
abspath = "//?/" + abspath;
}
std::wstring wpath(UTF_8::convert_to_utf16(abspath));
std::replace(wpath.begin(), wpath.end(), '/', '\\');
DWORD rv = GetFullPathNameW(wpath.c_str(), 32767, resolved, NULL);
DWORD rv = GetFullPathNameW(wpath.c_str(), 32767, &resolved[0], NULL);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
DWORD rv = GetFullPathNameW(wpath.c_str(), 32767, &resolved[0], NULL);
DWORD rv = GetFullPathNameW(wpath.c_str(), max_chars, resolved.data(), NULL);

src/file.cpp Show resolved Hide resolved
// windows unicode filepaths are encoded in utf16
std::string abspath(join_paths(get_cwd(), path));
if (!(abspath[0] == '/' && abspath[1] == '/')) {
abspath = "//?/" + abspath;
}
std::wstring wpath(UTF_8::convert_to_utf16(abspath));
std::replace(wpath.begin(), wpath.end(), '/', '\\');
DWORD rv = GetFullPathNameW(wpath.c_str(), 32767, resolved, NULL);
DWORD rv = GetFullPathNameW(wpath.c_str(), 32767, &resolved[0], NULL);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
DWORD rv = GetFullPathNameW(wpath.c_str(), 32767, &resolved[0], NULL);
DWORD rv = GetFullPathNameW(wpath.c_str(), max_chars, resolved.data(), NULL);

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't be necessary, since GetFullPathNameW will fill the buffer anyway!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This suggestion is to avoid repeating 32767 and to use .data() instead of &[0]

if (rv > 32767) throw Exception::OperationError("Path is too long");
if (rv == 0) throw Exception::OperationError("Path could not be resolved");
HANDLE hFile = CreateFileW(resolved, GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL);
HANDLE hFile = CreateFileW(&resolved[0], GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
HANDLE hFile = CreateFileW(&resolved[0], GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL);
HANDLE hFile = CreateFileW(resolved.data(), GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, 0, NULL);

if (rv > 32767) throw Exception::OperationError("Path is too long");
if (rv == 0) throw Exception::OperationError("Path could not be resolved");
DWORD dwAttrib = GetFileAttributesW(resolved);
DWORD dwAttrib = GetFileAttributesW(&resolved[0]);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
DWORD dwAttrib = GetFileAttributesW(&resolved[0]);
DWORD dwAttrib = GetFileAttributesW(resolved.data());

@@ -437,18 +437,18 @@ namespace Sass {
#ifdef _WIN32
BYTE* pBuffer;
DWORD dwBytes;
wchar_t resolved[32768];
std::vector<wchar_t> resolved(32768);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
std::vector<wchar_t> resolved(32768);
const std::size_t max_chars = 32767;
std::vector<wchar_t> resolved(max_chars + 1, 0);

@mgreter
Copy link
Contributor Author

mgreter commented Oct 27, 2019

@glebm care to create a PR with your suggested changes!?

@saper
Copy link
Member

saper commented Oct 28, 2019

Stack allocation is free but heap isn't; how is this an optimization?

Out of stack issues? https://docs.microsoft.com/en-us/visualstudio/code-quality/c6262?view=vs-2019

There is nothing wrong per se in having 32KB on stack. We could add an option to have /analyze:stacksize40000 or something.

Did this cause real-life crashes of libsass due to running out of stack? Even Alpine Linux lets us having more stack ... (but this code does not apply to Linux).

@mgreter
Copy link
Contributor Author

mgreter commented Jan 17, 2020

The memory payload is actually 64k, since the allocated array is of wchar.
In the end this is just a precaution to satisfy MSVC static code analysis.
But I think it does make sense to allocate those 64k on the heap!?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants