What's the proper way to protect dmu_buf_impl.db.db_data?

[asomers]

According to the comments in dbuf.h, db.db_data is protected by db_mtx, while db_buf is protected by db_rwlock. That's been the case ever since f664f1e . However, the code seems a little bit inconsistent. Here's a table of a few selected functions, and which fields they protect with which locks.

	db_buf/db_mtx	db_buf/db_rwlock	db.db_data/db_mtx	db.db_data/db_rwlock	db.db_data/nothing
dbuf_clear_data	x		x
dbuf_set_data	x		x
dbuf_loan_arcbuf	x				x
dbuf_read_bonus			x
dbuf_handle_indirect_hole					x
dbuf_read_hole			x
dbuf_verify				x
dbuf_free_range	x			x
dbuf_find_bp				x
dbuf_hold_copy				x
dbuf_write_ready			x

I can't find any examples of a function that takes db_rwlock just to protect db_mtx. I think that what should be happening is that db_buf is protected by db_mtx, and db.db_data should be protected by db_rwlock (or perhaps protected by both?). Can anybody please help me understand this? It's not merely an academic exercise either. I suspect that locking mistakes on db.db_data are responsible for #16626 and the corruption described in #17077 . cc @pcd1193182

[asomers]

Some possibilities for the writer that is racing with dbuf_write_ready are below. These functions write to db.db_data without locking db.db_mtx. The comment in dbuf.h still says that db_data "is protected by db_mtx", without any conditions, so I don't know why these functions don't lock it:

• dnode_increase_indirection
• dbuf_write_children_ready
• free_children