pyspark.sql.datasource.DataSourceStreamReader.getDefaultReadLimit

DataSourceStreamReader.getDefaultReadLimit()

Returns the read limits potentially passed to the data source through options when creating the data source. See the built-in implementations of ReadLimit for available read limits.

Implementing this method is optional. By default, it returns ReadAllAvailable, which means there is no limit on the amount of data returned by latestOffset().

New in version 4.2.0.