Paginate all vpc calls. #48
Conversation
|
Should replace #43 (I just couldn't manage to update the other PR) |
vpc.go
Outdated
| } | ||
| var unexpectedNumberOfVpcs error | ||
| var cidrBlockAssociations []*ec2.VpcCidrBlockAssociation | ||
| err := c.ec2.DescribeVpcsPagesWithContext(ctx, input, func(page *ec2.DescribeVpcsOutput, lastPage bool) bool { |
There was a problem hiding this comment.
I guess pagination isn't really needed when we only want to get a single vpc. I'd rather leave this function unchanged as this makes it way harder to understand what it's doing.
vpc.go
Outdated
| }}, | ||
| }}} | ||
| err := c.ec2.DescribeRouteTablesPagesWithContext(ctx, input, func(page *ec2.DescribeRouteTablesOutput, lastPage bool) bool { | ||
| routeTables = append(routeTables, page.RouteTables...) |
There was a problem hiding this comment.
As we're only interested in the number of tables, we can just count them instead of saving them all:
numRouteTables += len(page.RouteTables)
vpc.go
Outdated
|
|
||
| var vpcEndpoints []*ec2.VpcEndpoint | ||
| err := c.ec2.DescribeVpcEndpointsPagesWithContext(ctx, descEndpointsInput, func(page *ec2.DescribeVpcEndpointsOutput, lastPage bool) bool { | ||
| vpcEndpoints = append(vpcEndpoints, page.VpcEndpoints...) |
There was a problem hiding this comment.
As we only care about the number, i would not save all the individual endpoints:
numEndpoints += len(page.VpcEndpoints)
vpc.go
Outdated
| RouteTableIds: []*string{rtb.RouteTableId}, | ||
| }, func(page *ec2.DescribeRouteTablesOutput, lastPage bool) bool { |
There was a problem hiding this comment.
I find it quite awkward to do pagination on this call, as we're describing only a single routetable. It makes the function much harder to understand.
There was a problem hiding this comment.
In the unlikely case that the api call returns multiple pages, we error out anyway
vpc.go
Outdated
| describeVpcsOutput, err := c.ec2.DescribeVpcsWithContext(ctx, &ec2.DescribeVpcsInput{}) | ||
| var vpcs []*ec2.Vpc | ||
| err := c.ec2.DescribeVpcsPagesWithContext(ctx, &ec2.DescribeVpcsInput{}, func(page *ec2.DescribeVpcsOutput, lastPage bool) bool { | ||
| vpcs = append(vpcs, page.Vpcs...) |
There was a problem hiding this comment.
Also here, i'd suggest to only save the actual number of vpcs, and not the vpcs to make it clearer what we are attempting to do.
vpc.go
Outdated
| Name: aws.String("vpc-id"), | ||
| Values: []*string{vpc.VpcId}, | ||
| }}, | ||
| }}}, func(page *ec2.DescribeSubnetsOutput, lastPage bool) bool { | ||
| subnets = append(subnets, page.Subnets...) |
There was a problem hiding this comment.
Same here, i'd suggest to only save the actual number of subnets, and not the actual subnets so it's easier to understand what we are attempting.
bergmannf
force-pushed
the
paginate-vpc
branch
from
4177649 to
3d42def
Compare
bergmannf
force-pushed
the
paginate-vpc
branch
from
3d42def to
be88092
Compare
bergmannf
force-pushed
the
paginate-vpc
branch
from
be88092 to
1cd741d
Compare
Use pagination for all vpc metrics, so we don't miss out on any data.