Security

AD is only returning 1000 entries. How do I get around this?

the_wolverine
Champion

I'm trying to configure LDAP auth for Splunk. I'm running into an issue where AD is only giving me 1000 entries and the users I'm looking for are not being returned. How do I get around this?

Tags (2)
1 Solution

the_wolverine
Champion

By default, AD imposes a sizelimit of 1000 entries. This is the max number of entries that can be returned in a single bind request. There is no way to modify this (outside of AD) so we must use userBaseFilters and/or groupBaseFilters to reduce the number of entries returned.

A separate call is made on userBaseDN and groupBaseDN so the max is really 1000 user entries and 1000 group entries PER DN. If you specify multiple DNs in your configuration, multiple calls are made to AD, each which allows the max sizelimit.

A workaround?

  • Try using a more specific BaseDN. For example, instead of dc=SplunkSupport,dc=Com, use a combination of : ou=People,ou=USA,dc=SplunkSupport,dc=Com;ou=People,ou=Canada,dc=SplunkSupport,dc=Com;

  • Use LDAP filters. See this blog post for examples of LDAP filters used to further reduce the number of entries returned.

  • Use the AD GC port.

View solution in original post

matt_1
Explorer

I actually ended up having to specify each user's sAMAccountName within the "userBaseFilter". Fortunately I was able to write a script to generate groupBaseDN, groupBaseFilter, userBaseDN, and userBaseFilter by wrapping the Solaris ldapsearch.

The goal is to automate that portion so that Splunk access can be inherited through the AD Group, distribution list, or mailbox member.

Here is the pilot script. The final one should actually build the authentication.conf, reload auth, set user defaults relative to roles (i.e. set the default application relative to user and role), and be managed through a configuration file. Use Splunk cron scheduler to run once or twice a week.

#!/usr/bin/perl

my(@domains) = (
  '<domain label 1> ',
  '<domain label 2> ',
  '<domain label n> ',
);

my(%cfgs);
if(@ARGV)
{
  push(@{$cfgs{adGroups}},@ARGV);
}
else
{
  @{$cfgs{adGroups}} = (
    '<AD Group sAMAccountName 1>',
    '<AD Group sAMAccountName 2>',
    '<AD Group sAMAccountName n>',
  );
}

%{$cfgs{adBindParameters}} = (
  '<domain label 1>' => {
    'cn' => 'CN=<...>,OU=<...>,DC=<....>,DC=com',
    'pw' => '<Bind Password>',
    'dc' => '<Domain Controller or LDAP Host>',
    'pt' => '<port>',
    'tr' => '<Top OU of Search Path>'
  },
  '<domain label 2>' => {
    ...........
  }
  '<domain label n>' => {
    ...........
  }
);

my($regex) = '^CN=[^,]+,[^,]+,\s*?(OU\=[^$]+)$';
foreach my $domain (@domains)
{
  print "##\n## [[$domain]]----------------------------\n##\n";
  my(@userBaseDN);
  my(@groupBaseDN);
  my($groupList);
  my($userBaseFilter) = '(|';
  my($groupBaseFilter) = '(|';
  foreach my $group (@{$cfgs{adGroups}})
  {
    my($memberList);
    my($cmd) = (
      '/usr/bin/ldapsearch' .
      ' -v' .
      ' -x' .
      ' -h "' . $cfgs{adBindParameters}{$domain}{dc} . '"' .
      ' -p "' . $cfgs{adBindParameters}{$domain}{pt} . '"' .
      ' -b "' . $cfgs{adBindParameters}{$domain}{tr} . '"' .
      ' -D "' . $cfgs{adBindParameters}{$domain}{cn} . '"' .
      ' -w "' . $cfgs{adBindParameters}{$domain}{pw} . '"' .
      ' "(sAMAccountName=' . $group . ')"' .
      ' -W "member"'
    );

    #--------------------------------------------------------------#
    # Grab DN for each group member and DN for evaluated AD group. #
    #--------------------------------------------------------------#
    if(open(adInfo,"$cmd|"))
    {
      my($memberJoinFlag);
      my($groupJoinFlag);
      while(<adInfo>)
      {
        s/^\s+|\s+$//g;
        s/\\/\\\\\\\\/gm;
        if(/^member:\s*(.*)/)
        {
          $memberJoinFlag = 1;
          $memberList .= "\n$1";
        }
        elsif(/^dn:\s*(.*)/)
        {
          $groupJoinFlag = 1;
          $groupList .= "\n$1";
        }
        elsif($memberJoinFlag && !/^[\d]+\smatches/)
        {
          $memberJoinFlag;
          $memberList .= $_;
        }
        elsif($groupJoinFlag && !/^[\d]+\smatches/)
        {
          $groupJoinFlag;
          $groupList .= $_;
        }
      }
      close(adInfo);
    }

    #---------------------------------------------------------------#
    # Determine each member's UID (sAMAAccountName) from DN.  Build #
    # 'userBaseFilter and 'groupBaseFilter'                         #
    #---------------------------------------------------------------#
    my(@memberDNs) = split("\n",$memberList);
    my($duplicate);
    foreach my $member (@memberDNs)
    {
      if($member =~ /$regex/)
      {
        my($ou) = $1;
        my($dupeKey) = $ou;
        $dupeKey =~ s/[\W]//gm;

        unless(defined $duplicate{$dupeKey})
        {
          push(@userBaseDN,"$ou");
        }
        $duplicate{$dupeKey} = 1;
      }

      $cmd = (
        '/usr/bin/ldapsearch' .
        ' -v' .
        ' -x' .
        ' -h "' . $cfgs{adBindParameters}{$domain}{dc} . '"' .
        ' -p "' . $cfgs{adBindParameters}{$domain}{pt} . '"' .
        ' -b "' . $cfgs{adBindParameters}{$domain}{tr} . '"' .
        ' -D "' . $cfgs{adBindParameters}{$domain}{cn} . '"' .
        ' -w "' . $cfgs{adBindParameters}{$domain}{pw} . '"' .
        ' "(distinguishedName=' . $member . ')"' .
        ' -W "sAMAccountName"'
      );
      if(open(adInfo,"$cmd|"))
      {
        while(<adInfo>)
        {
          s/^\s+|\s+$//g;
          if(/^sAMAccountName:\s*([\w]+)$/)
          {
            my($userAcctName) = uc($1);
            if(
              ($userAcctName =~ /^[aAuU][aA-zZ,0-9]{6}$/) &&
              (!defined $duplicate{$userAcctName})
            )
            {
              $userBaseFilter .= "(sAMAccountName=$userAcctName)";
            }

            unless(defined $duplicate{$group})
            {
              $groupBaseFilter .= "(sAMAccountName=$group)";
            }

            $duplicate{$userAcctName} = 1;
            $duplicate{$group} = 1;
          }
        }
        close(adInfo);
      }
    }
  }
  $groupBaseFilter .= ')';
  $userBaseFilter  .= ')';

  #-----------------------------------------------------------#
  # Determine a list of all OU search paths for "groupBaseDN" #
  #-----------------------------------------------------------#
  my(@groupDNs) = split("\n",$groupList);
  my(%duplicate);
  foreach(@groupDNs)
  {
    if(/$regex/)
    {
      my($ou) = $1;
      my($dupeKey) = $ou;
      $dupeKey =~ s/[\W]//gm;

      unless(defined $duplicate{$dupeKey})
      {
        push(@groupBaseDN,"$ou");
      }
      $duplicate{$dupeKey} = 1;
    }
  }

  print "groupBaseDN = " . join(';',@groupBaseDN) . "\n\n";
  print "groupBaseFilter = $groupBaseFilter\n\n";
  print "userBaseDN = "  . join(';',@userBaseDN) . "\n\n";
  print "userBaseFilter = $userBaseFilter\n\n";
}


__END__
0 Karma

the_wolverine
Champion

By default, AD imposes a sizelimit of 1000 entries. This is the max number of entries that can be returned in a single bind request. There is no way to modify this (outside of AD) so we must use userBaseFilters and/or groupBaseFilters to reduce the number of entries returned.

A separate call is made on userBaseDN and groupBaseDN so the max is really 1000 user entries and 1000 group entries PER DN. If you specify multiple DNs in your configuration, multiple calls are made to AD, each which allows the max sizelimit.

A workaround?

  • Try using a more specific BaseDN. For example, instead of dc=SplunkSupport,dc=Com, use a combination of : ou=People,ou=USA,dc=SplunkSupport,dc=Com;ou=People,ou=Canada,dc=SplunkSupport,dc=Com;

  • Use LDAP filters. See this blog post for examples of LDAP filters used to further reduce the number of entries returned.

  • Use the AD GC port.

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...