Residential Whole Loans - Realized Credit Losses Net of Recoveries (Details) - Residential whole loans - USD ($) $ in Thousands	3 Months Ended		6 Months Ended
	Jun. 30, 2025	Jun. 30, 2024	Jun. 30, 2025	Jun. 30, 2024
Accounts, Notes, Loans and Financing Receivable [Line Items]
Realized credit losses, net of recoveries	$ (9,812)	$ (1,349)	$ (13,542)	$ (1,991)
Non-QM loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Realized credit losses, net of recoveries	(649)	87	(785)	88
Business purpose loans \| Single-family rental loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Realized credit losses, net of recoveries	(3,148)	(567)	(4,344)	(1,454)
Business purpose loans \| Single-family transitional loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Realized credit losses, net of recoveries	(1,351)	(271)	(3,524)	381
Business purpose loans \| Multifamily transitional loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Realized credit losses, net of recoveries	(4,080)	(80)	(4,249)	(107)
Legacy RPL/NPL loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Realized credit losses, net of recoveries	(584)	(518)	(640)	(899)
Other loans
Accounts, Notes, Loans and Financing Receivable [Line Items]
Realized credit losses, net of recoveries	$ 0	$ 0	$ 0	$ 0