7 min read ASP.NET Core

Implementing Paging for Large Datasets in .NET Applications

Today, we're diving into a crucial aspect of building responsive and performant applications: **paging**.
Implementing Paging for Large Datasets in .NET Applications

Hey everyone! Today, we're diving into a crucial aspect of building responsive and performant applications: paging. If you're dealing with datasets that can grow large – think lists of countries, hotels, bookings, or any other extensive collection – loading everything at once is a recipe for disaster. It's slow, it hogs resources, and it makes for a terrible user experience. That's where paging comes in, and we're going to explore how to implement it effectively in our .NET applications.

🎥 Prefer to follow along visually?
This article is based on a full walkthrough video where I implement paging step-by-step in an ASP.NET Core Web API using .NET 10 and Entity Framework Core.

👉 Watch the full video here: https://youtu.be/serGD6MdoWE

What is Paging and Why Do We Need It?

At its core, paging is a technique for breaking down large chunks of data into smaller, more manageable pieces, called "pages." Instead of fetching thousands of records in a single go, we retrieve them in smaller batches.

Why is this so important?

  • Performance Boost: Loading a subset of data is significantly faster than loading an entire dataset. This means quicker load times for your users.
  • Reduced Network Traffic: Sending less data over the network reduces bandwidth consumption and makes the application more efficient.
  • Improved User Experience: Applications feel snappier and more responsive when data appears quickly, rather than making users wait for a massive download.
  • Resource Management: Less strain on both the server and the client's memory and processing power.

Imagine trying to load 10,000 hotel records all at once. It's simply not smart! Paging allows us to load just a handful at a time, making our applications much more user-friendly and efficient.

Setting Up Our Paging Models

To implement paging, we need a few key components. We'll start by defining the parameters a user might send to request specific pages, and by defining a model to encapsulate the paged results.

PaginationParameters Model

This model will define the rules for how users can request data. We'll create a new class in our Common/Models folder, and to keep things organized, we'll create a subfolder named Paging.

// Common/Models/Paging/PaginationParameters.cs
public class PaginationParameters
{
    public const int MaxPageSize = 50; // Maximum number of records per page
    public int PageNumber { get; set; } = 1; // Default to page 1

    private int _pageSize = 10; // Default page size

    public int PageSize
    {
        get => _pageSize;
        set => _pageSize = (value > MaxPageSize) ? MaxPageSize : value; // Ensure page size doesn't exceed max
    }
}

In this PaginationParameters class:

  • We define a MaxPageSize constant to set a hard limit of 50 records per page. This prevents users from requesting an unreasonably large number of items.
  • The PageNumber defaults to 1, meaning if no page number is specified, we'll start with the first page.
  • The PageSize also defaults to 10. The setter for PageSize includes logic to cap it at MaxPageSize. This means even if a user requests more than 50 items, they'll only get 50.

We also add range validation to ensure the PageNumber is at least 1 and the PageSize is between 1 and MaxPageSize.

PagedResult<T> Model

Next, we need a way to return not only the data but also information about the pagination. This is where our PagedResult<T> model comes in. It will hold the actual data and the pagination metadata.

// Common/Models/Paging/PagedResult.cs
public class PagedResult<T>
{
    public List<T> Data { get; set; }
    public PaginationMetadata Metadata { get; set; }
}

And the PaginationMetadata model will look like this:

// Common/Models/Paging/PaginationMetadata.cs
public class PaginationMetadata
{
    public int CurrentPage { get; set; }
    public int PageSize { get; set; }
    public int TotalCount { get; set; }
    public int TotalPages { get; set; }
    public bool HasNextPage { get; set; }
    public bool HasPreviousPage { get; set; }
}

This PaginationMetadata provides context: which page we're on, how many items are on that page, the total number of items available, the total number of pages, and whether there's a next or previous page to navigate to.

Creating an Extension Method for Paging

To avoid repeating the paging logic every time we query data, we'll create an extension method for IQueryable<T>. This is a fantastic way to add custom functionality to existing types without modifying their original code.

We'll create a new folder called Extensions within our Common/Models directory and add a class named QueryableExtensions.

// Common/Models/Extensions/QueryableExtensions.cs
using Microsoft.EntityFrameworkCore; // Assuming EF Core for ToListAsync
using System.Linq;
using System.Threading.Tasks;

public static class QueryableExtensions
{
    public static async Task<PagedResult<T>> ToPagedResultAsync<T>(
        this IQueryable<T> source,
        PaginationParameters paginationParameters)
    {
        // Get total count of records
        var totalCount = await source.CountAsync();

        // Apply skip and take for paging
        var items = await source
            .Skip((paginationParameters.PageNumber - 1) * paginationParameters.PageSize)
            .Take(paginationParameters.PageSize)
            .ToListAsync();

        // Calculate total pages
        var totalPages = (int)Math.Ceiling((double)totalCount / paginationParameters.PageSize);

        // Construct pagination metadata
        var metadata = new PaginationMetadata
        {
            CurrentPage = paginationParameters.PageNumber,
            PageSize = paginationParameters.PageSize,
            TotalCount = totalCount,
            TotalPages = totalPages,
            HasNextPage = paginationParameters.PageNumber < totalPages,
            HasPreviousPage = paginationParameters.PageNumber > 1
        };

        // Return the paged result
        return new PagedResult<T>
        {
            Data = items,
            Metadata = metadata
        };
    }
}

Let's break down this extension method:

  1. public static class QueryableExtensions: Extension methods must reside in static classes.
  2. public static async Task<PagedResult<T>> ToPagedResultAsync<T>(this IQueryable<T> source, PaginationParameters paginationParameters):
    • this IQueryable<T> source: This is how we define that this method extends IQueryable<T>. source will be the IQueryable collection we call this method on.
    • PaginationParameters paginationParameters: This parameter contains the page number and page size requested by the client.
    • async Task<PagedResult<T>>: We're returning a PagedResult asynchronously.
  3. var totalCount = await source.CountAsync();: We first get the total number of records in the source IQueryable.
  4. .Skip((paginationParameters.PageNumber - 1) * paginationParameters.PageSize): This calculates how many records to skip. For page 1, we skip 0. For page 2, we skip PageSize records, and so on.
  5. .Take(paginationParameters.PageSize): This specifies how many records to take after skipping.
  6. .ToListAsync(): Executes the query and fetches the subset of data.
  7. var totalPages = (int)Math.Ceiling((double)totalCount / paginationParameters.PageSize);: We calculate the total number of pages by dividing the total count by the page size and rounding up using Math.Ceiling.
  8. Constructing PaginationMetadata: We populate the metadata object with all relevant information.
  9. Returning PagedResult<T>: Finally, we create and return our PagedResult object containing the fetched data and its associated metadata.

🔎 Deep dive note
If you want to see this paging extension implemented and wired up live—along with explanations of why each decision was made—I walk through this exact code in detail in the companion video.

👉 Watch the walkthrough: https://youtu.be/serGD6MdoWE

Integrating Paging into Services and Controllers

Now that we have our paging infrastructure in place, let's see how to use it in our application.

Refactoring Service Methods

Consider a service method that returns a collection, like GetBookingsForHotelAsync. We'll change its return type to Task<PagedResult<BookingDto>> and use our new extension method.

We'll also need to add the PaginationParameters to the method signature.

// Example in a BookingService
public async Task<Result<PagedResult<BookingDto>>> GetBookingsForHotelAsync(
    int hotelId, PaginationParameters paginationParameters)
{
    // ... hotel existence check ...

    var bookings = await _context.Bookings
        .Where(b => b.HotelId == hotelId)
        .ProjectTo<BookingDto>(_mapper.ConfigurationProvider) // Assuming AutoMapper
        .ToPagedResultAsync(paginationParameters); // Using our extension method

    return Result<PagedResult<BookingDto>>.Success(bookings);
}

Notice how we replaced .ToListAsync() with .ToPagedResultAsync(paginationParameters). This makes the integration incredibly clean.

Updating Controllers

In our controllers, we'll need to do a couple of things:

  1. Change the return type of the endpoint to ActionResult<PagedResult<BookingDto>>.
  2. Add PaginationParameters to the action method parameters. These will typically come from the query string.
  3. Pass these parameters to the service method.
// Example in HotelBookingsController
[HttpGet]
public async Task<ActionResult<PagedResult<BookingDto>>> GetBookingsForHotel(
    int hotelId, [FromQuery] PaginationParameters paginationParameters)
{
    var result = await _bookingService.GetBookingsForHotelAsync(hotelId, paginationParameters);

    if (!result.IsSuccess)
    {
        return BadRequest(result.Error);
    }

    return Ok(result.Value);
}

When a request comes in, for example, /api/hotels/1/bookings?pageNumber=2&pageSize=20, the hotelId will be 1, paginationParameters.PageNumber will be 2, and paginationParameters.PageSize will be 20. These will then be passed down to our service and our extension method.

Testing Our Paged Results

Let's imagine we've made a request to retrieve hotel bookings and included the query parameters for paging. The JSON response might look something like this:

{
  "data": [
    // Array of BookingDto objects for the current page
    {
      "id": 101,
      "guestName": "Alice Smith",
      // ... other booking details
    },
    {
      "id": 102,
      "guestName": "Bob Johnson",
      // ... other booking details
    }
  ],
  "metadata": {
    "currentPage": 1,
    "pageSize": 10,
    "totalCount": 25, // Total number of bookings for this hotel
    "totalPages": 3,  // 25 total / 10 per page = 2.5, rounded up to 3
    "hasNextPage": true,
    "hasPreviousPage": false
  }
}

As you can see, we retrieve the actual data for the current page, and the metadata clearly indicates our current position within the entire dataset, as well as navigation options. If currentPage was 3, hasNextPage would be false and hasPreviousPage would be true.

Conclusion

Implementing paging is a fundamental step towards building robust and scalable applications. By introducing dedicated models such as PaginationParameters and PagedResult<T>, and leveraging extension methods like ToPagedResultAsync, we can efficiently manage large datasets, improve performance, and deliver a superior user experience. This approach keeps our code clean, organized, and easy to maintain.

Want to Take This Further?

Paging is just one small but important part of building a production-ready ASP.NET Core Web API.

In my full course, Ultimate ASP.NET Core Web API Development Guide, we go far beyond paging and build a complete, enterprise-ready API using .NET 10 and modern best practices.

Inside the course, you will learn how to:

  • Design clean, RESTful APIs end-to-end
  • Combine filtering, sorting, and paging correctly
  • Structure services, repositories, and DTOs
  • Secure APIs with authentication and JWT
  • Add logging, documentation, caching, and versioning
  • Deploy APIs and databases to Microsoft Azure

👉 Enroll here:
Ultimate ASP.NET Core Web API Development Guide

Happy coding!

Published by:

Trevoir Williams

Member discussion