Add Scroll only after X amount of ng-repeat items

I have a list of cards, and each card has users who have taken that action. I would like to scroll horizontally through the users only if there are more than 4 users on that card, otherwise I would like to have scroll disabled.

This is what each card looks like.

Here is the code I have for the users area:

<ion-scroll delegate-handle="{{action._id}}" direction="x" scrollbar-x="false" has-bouncing="false" overflow-scroll="true">
           <div class="action-avatar-row flex flex-dir-row">
                    <div class="action-avatar-col" ng-repeat="user in action.users | limitTo:20 | reverse">
                           <div class="action-card-avatar-outline" style="background-color: {{user.color}}">
                                    <img src="{{user.avatar}}" class="action-card-avatar">

As of right now, the users area scrolls even with just one user.

I have tried ion-content and ion-scroll. ion-content wouldn’t allow the scroll on the device, yet worked in Chrome. I have also tried $ionicScrollDelegate.resize, with no luck. Any help would be awesome!