Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Persist pgstat file to preserve statistic between sessions #357

Open
wants to merge 66 commits into
base: REL_15_STABLE_neon
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
b2f37fc
Neon rebased to REL_15_STABLE (up to 0a9045c9ff)
lubennikovaav Jul 22, 2022
58cfeeb
fix regression tests
lubennikovaav Aug 10, 2022
65dd0f7
rebase to the latest origin and resolve conflicts
lubennikovaav Aug 16, 2022
b2dbd9d
Remove contrib neon and neon_test_utils.
lubennikovaav Aug 24, 2022
283fcdf
Prevent access to uninitalized shaerd memory in InstallXLogFileSegmen…
lubennikovaav Aug 24, 2022
205d59d
Remove Dockerfile, it's now in the neon repo
lubennikovaav Aug 29, 2022
246bae1
Move backpressure throttling implementation to neon extension (#203)
knizhnik Sep 2, 2022
ad18f68
Merge last written cache lsn with new main branch (#201)
knizhnik Sep 4, 2022
9b168ed
Local prefetch implementation for Postgres 15
hlinnaka Sep 13, 2022
5237824
Set last written LSN for the created relation (#212)
hlinnaka Sep 16, 2022
871400c
Use normal install program to install server headers.
hlinnaka Sep 12, 2022
945737b
Update expected output for sysviews test because of changed default v…
knizhnik Sep 12, 2022
3c976a2
Undo diasming VM check warning in vacuumlazy.c (#214)
knizhnik Sep 20, 2022
67d1c0a
Set Neon-specific FMGR_ABI_EXTRA
lubennikovaav Sep 21, 2022
c85bc09
Don't use newline in PG_VERSION file.
lubennikovaav Sep 21, 2022
d363e94
Unset ArchiveRecoveryRequested for Neon code path.
lubennikovaav Sep 21, 2022
eefa704
Fix memory leak in ApplyRecord
knizhnik Sep 21, 2022
ed3602e
Rebase to Stamp 15.0
lubennikovaav Oct 5, 2022
f50d2de
Pin pages with speculative insert tuples to prevent their reconstruct…
knizhnik Oct 20, 2022
2df27d7
Fix shared memory initialization for last written LSN cache (#226)
knizhnik Oct 24, 2022
7c26185
Fix upper boundary caculation in the chunks loop in SetLastWrittenLSN…
knizhnik Oct 28, 2022
ea88e63
Stamp XLP_FIRST_IS_CONTRECORD only if we start writing with page offset.
arssher Oct 28, 2022
ae333c0
Move walredo process code under pgxn in the main 'neon' repository.
hlinnaka Oct 31, 2022
7ab6932
Misc cleanup, mostly to reduce unnecessary differences with upstream.
hlinnaka Oct 31, 2022
7e5a1f0
Optimize prefetch patterns in both heap seqscan and vacuum scans. (#228)
MMeent Nov 7, 2022
9544eaa
Fix prefetch issues in parallel scans and vacuum's cleanup scan (#235)
MMeent Nov 11, 2022
3a06992
Fix expected results for regression tests (#238)
bayandin Nov 15, 2022
4d4f707
Use prefetch in pg_prewarm extension (#237)
knizhnik Nov 17, 2022
d2453bd
PG15: Prefetch cleanup (#241)
MMeent Nov 23, 2022
ebf2fec
Drop unlogged table in regress test to avoid noise in tests
lubennikovaav Nov 21, 2022
0d5c7be
Do not produce open file error for unlogged relations (#240)
knizhnik Nov 24, 2022
9bcaa14
Maintain last written LSN for each page to enable prefetch on vacuum,…
knizhnik Nov 24, 2022
3ace5a3
Prefetch cleanup: (#246)
MMeent Dec 5, 2022
20d665d
Set lsn fix v15 (#252)
knizhnik Dec 8, 2022
8c1ae26
Show prefetch statistic in EXPLAIN (#249)
knizhnik Jan 12, 2023
723f85f
Implement efficient prefetch for parallel bitmap heap scan (#258)
knizhnik Feb 14, 2023
1621daf
Unlogged index fix v15 (#262)
knizhnik Feb 22, 2023
1c049dc
Fix bitmap scan prefetch (#261)
knizhnik Feb 23, 2023
d1e3fb3
Revert handling of UNLOGGED tables on compute side v15.
arssher Feb 24, 2023
a9a9b43
Allow external main functions to skip config load and make last
kelvich Feb 23, 2023
c2b4266
Remove walredo-related hacks from InternalIpcMemoryCreate()
kelvich Mar 6, 2023
8fea298
Adjust prefetch target for parallel bitmap scan (#274)
knizhnik Mar 27, 2023
59f7ecb
Copy iterator result in BitmapHeapNext (#276)
knizhnik Apr 7, 2023
34608d6
Prefetch for index and index-only scans (#271)
knizhnik Apr 7, 2023
96fca0d
[PG15] Feature/replicas (#279)
MMeent Apr 13, 2023
e3e5075
Fix entering hot standby mode for Neon
lubennikovaav May 5, 2023
875fc76
Do not allow users with CREATEROLE privelege to manage system user
kelvich Jul 7, 2023
58b7244
Fix regression tests after the patch with CREATEROLE restrictions
kelvich Jul 15, 2023
b3b692c
Add startup logs (#293)
bojanserafimov Jul 17, 2023
ac82176
Make it possible to grant self created roles (#298)
knizhnik Jul 27, 2023
c6c5d17
Update expected file for create_role test (#301)
knizhnik Jul 28, 2023
429ecaf
Define NEON_SMGR in smgr.h to make it possible for extensions to use …
knizhnik Jul 29, 2023
9e1dd46
Request extension files and libraries from compute_ctl
lubennikovaav Jun 13, 2023
1d89f6d
Upadte last written LSN after walloging all createdb stuff (#307)
knizhnik Sep 25, 2023
1afb4f6
[refer #111] Persist logical replication files in WAL (#306)
knizhnik Oct 18, 2023
9ca7417
Fix elog format error in wallog_mapping_file (#316)
knizhnik Oct 19, 2023
ac718a9
Update WAL buffers when restoring WAL at compute needed for LR (#324)
knizhnik Nov 3, 2023
98d8d6a
Prevent output callbacks from hearing about neon-file messages (#329)
Dec 14, 2023
b9591d3
strncmp vs strcmp
save-buffer Dec 15, 2023
b0297cd
Remove excessive walsender reply logging.
arssher Oct 19, 2023
940d29b
Allow publications FOR ALL TABLES to neon_superuser
save-buffer Jan 8, 2024
8d18a8d
Switch GetCurrentRoleId to GetUserId
save-buffer Jan 24, 2024
0ff55c5
Support creating subscriptions as neon_superuser
Jan 25, 2024
6ee78a3
On demand downloading of SLRU segments (#332)
knizhnik Jan 31, 2024
771e5f5
Persist pgstat file to preserve statistic between sessions
Feb 8, 2024
6afa5f4
Persistent pgstat file only if it is successfully created
Feb 9, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Stamp XLP_FIRST_IS_CONTRECORD only if we start writing with page offset.
Without this patch, on bootstrap XLP_FIRST_IS_CONTRECORD has been always put on
header of a page where WAL writing continues. This confuses WAL decoding on
safekeepers, making it think decoding starts in the middle of a record, leading
to

 2022-08-12T17:48:13.816665Z ERROR {tid=37}: query handler for 'START_WAL_PUSH postgresql://no_user:@localhost:15050' failed: failed to run ReceiveWalConn

 Caused by:
    0: failed to process ProposerAcceptorMessage
    1: invalid xlog page header: unexpected XLP_FIRST_IS_CONTRECORD at 0/2CF8000

Rebase of a1af529 for v14.
  • Loading branch information
arssher authored and tristan957 committed Feb 6, 2024
commit ea88e6389848e707c8aa9483fc03710cb6247ac4
16 changes: 11 additions & 5 deletions src/backend/access/transam/xlogrecovery.c
Original file line number Diff line number Diff line change
Expand Up @@ -1570,17 +1570,23 @@ FinishWalRecovery(void)
}
else
{
int len = endOfLog % XLOG_BLCKSZ;
char *page = palloc0(len);
XLogRecPtr pageBeginPtr = endOfLog - (endOfLog % XLOG_BLCKSZ);
int offs = endOfLog % XLOG_BLCKSZ;
char *page = palloc0(offs);
XLogRecPtr pageBeginPtr = endOfLog - offs;
int lastPageSize = ((pageBeginPtr % wal_segment_size) == 0) ? SizeOfXLogLongPHD : SizeOfXLogShortPHD;

XLogPageHeader xlogPageHdr = (XLogPageHeader) (page);

xlogPageHdr->xlp_pageaddr = pageBeginPtr;
xlogPageHdr->xlp_magic = XLOG_PAGE_MAGIC;
xlogPageHdr->xlp_tli = recoveryTargetTLI;
xlogPageHdr->xlp_info = XLP_FIRST_IS_CONTRECORD; // FIXME
xlogPageHdr->xlp_rem_len = (endOfLog % XLOG_BLCKSZ) - SizeOfXLogShortPHD;
/*
* If we start writing with offset from page beginning, pretend in
* page header there is a record ending where actual data will
* start.
*/
xlogPageHdr->xlp_rem_len = offs - lastPageSize;
xlogPageHdr->xlp_info = (xlogPageHdr->xlp_rem_len > 0) ? XLP_FIRST_IS_CONTRECORD : 0;
readOff = XLogSegmentOffset(pageBeginPtr, wal_segment_size);

result->lastPageBeginPtr = pageBeginPtr;
Expand Down