| Author | Messages | |
mcasey
Posts:75
 | | 11/03/2008 11:33 AM |
| Hi All,
I am looking for feedback on indexing additional attributes in AD to
support an increased volume of LDAP queries. I am currently working
on several projects, including automated group management, that will
be leveraging data stored in user and group objects for various other
automated tasks. Some examples of attributes I currently use are;
employeeID, title, department, departmentNumber, employeeType, and a
few Exchange extensionAttributes.
What are some common justifications to index or not to index? It
would also be interesting to know what issues people have run into
when indexing various attributes in AD. The dit in my environment is
not that large, ~500MB, and space it not really a concern at this
point. To my knowledge we run only the default indexes for
AD(win2k3)/Exchange(2k3)/possibly Cisco(unity).
Thanks in advance!
-matt
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
| | | |
| bdesmond
Posts:977
| | 11/03/2008 12:10 PM |
| I'd index them and be done with it. Exchange 2007 setup indexes the extension attributes btw.
Thanks,
Brian Desmond
brian@briandesmond.com
c - 312.731.3132
-----Original Message-----
From: ActiveDir-owner@mail.activedir.org [mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Matt Casey
Sent: Monday, November 03, 2008 10:30 AM
To: ActiveDir@mail.activedir.org
Subject: [ActiveDir] Indexing
Hi All,
I am looking for feedback on indexing additional attributes in AD to
support an increased volume of LDAP queries. I am currently working
on several projects, including automated group management, that will
be leveraging data stored in user and group objects for various other
automated tasks. Some examples of attributes I currently use are;
employeeID, title, department, departmentNumber, employeeType, and a
few Exchange extensionAttributes.
What are some common justifications to index or not to index? It
would also be interesting to know what issues people have run into
when indexing various attributes in AD. The dit in my environment is
not that large, ~500MB, and space it not really a concern at this
point. To my knowledge we run only the default indexes for
AD(win2k3)/Exchange(2k3)/possibly Cisco(unity).
Thanks in advance!
-matt
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
| | | |
| DonH
Posts:65
| | 11/03/2008 2:53 PM |
| I'd recommend indexing them and then NOT being done with it. Instead, turn
up the logging level for searching (as shown in
http://msdn.microsoft.com/en-us/library/ms808539.aspx), run your queries,
and then at the results to see which of your newly added indices were
actually used satisfying your queries.
You're likely to find that indices on attributes with lots of unique values
(e.g., employeeID) are actually used, while indices on attributes with only
a few possible values (e.g., employeeType if that's just "full time" vs
"part time") are considered by the query processor but never used. You want
to keep the indices that the QP actually uses, and delete indices that are
not used.
Don
-----Original Message-----
From: ActiveDir-owner@mail.activedir.org
[mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Brian Desmond
Sent: Monday, November 03, 2008 9:06 AM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] Indexing
I'd index them and be done with it. Exchange 2007 setup indexes the
extension attributes btw.
Thanks,
Brian Desmond
brian@briandesmond.com
c - 312.731.3132
-----Original Message-----
From: ActiveDir-owner@mail.activedir.org
[mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Matt Casey
Sent: Monday, November 03, 2008 10:30 AM
To: ActiveDir@mail.activedir.org
Subject: [ActiveDir] Indexing
Hi All,
I am looking for feedback on indexing additional attributes in AD to
support an increased volume of LDAP queries. I am currently working
on several projects, including automated group management, that will
be leveraging data stored in user and group objects for various other
automated tasks. Some examples of attributes I currently use are;
employeeID, title, department, departmentNumber, employeeType, and a
few Exchange extensionAttributes.
What are some common justifications to index or not to index? It
would also be interesting to know what issues people have run into
when indexing various attributes in AD. The dit in my environment is
not that large, ~500MB, and space it not really a concern at this
point. To my knowledge we run only the default indexes for
AD(win2k3)/Exchange(2k3)/possibly Cisco(unity).
Thanks in advance!
-matt
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
| | | |
| Gil
Posts:311
| | 11/03/2008 3:15 PM |
| The only justification for indexing that I can think of is to improve query response time and to decrease load during those queries. The downsides are some increase in DIT size (usually not much) and increase in write times (again not much, and usually negligible in the grand scheme of things). I suppose that if the LSA working set grew just enough to cause a bunch of paging you could screw performance, but I've never seen (or heard of) that.
Make sure you pick the appropriate kind of index for your queries (DNT, PDNT, ANR, etc)
Indexes are good. Have some.
-g
-----Original Message-----
From: ActiveDir-owner@mail.activedir.org [mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Matt Casey
Sent: Monday, November 03, 2008 9:30 AM
To: ActiveDir@mail.activedir.org
Subject: [ActiveDir] Indexing
Hi All,
I am looking for feedback on indexing additional attributes in AD to
support an increased volume of LDAP queries. I am currently working
on several projects, including automated group management, that will
be leveraging data stored in user and group objects for various other
automated tasks. Some examples of attributes I currently use are;
employeeID, title, department, departmentNumber, employeeType, and a
few Exchange extensionAttributes.
What are some common justifications to index or not to index? It
would also be interesting to know what issues people have run into
when indexing various attributes in AD. The dit in my environment is
not that large, ~500MB, and space it not really a concern at this
point. To my knowledge we run only the default indexes for
AD(win2k3)/Exchange(2k3)/possibly Cisco(unity).
Thanks in advance!
-matt
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
| | | |
| dgavrilov
Posts:59
| | 11/03/2008 3:48 PM |
| One other thing to keep in mind is that regular indices are global, i.e. they index all the data in the DIT.
Thus, they are most useful when you do global searches (e.g. find a person in the domain or forest by his employeeID).
But if you run OU-scoped searches, then the query processor may choose not to use the global index, and resort to walking the OU hierarchy (ancestry index) instead.
Again, as Don recommends, do try to introduce the indices, and see if they are actually being utilized. It's best to do it on real (or close enough) data, and with real (or close enough) queries.
Dmitri
-----Original Message-----
From: ActiveDir-owner@mail.activedir.org [mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Don Hacherl
Sent: Monday, November 03, 2008 11:50 AM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] Indexing
I'd recommend indexing them and then NOT being done with it. Instead, turn
up the logging level for searching (as shown in
http://msdn.microsoft.com/en-us/library/ms808539.aspx), run your queries,
and then at the results to see which of your newly added indices were
actually used satisfying your queries.
You're likely to find that indices on attributes with lots of unique values
(e.g., employeeID) are actually used, while indices on attributes with only
a few possible values (e.g., employeeType if that's just "full time" vs
"part time") are considered by the query processor but never used. You want
to keep the indices that the QP actually uses, and delete indices that are
not used.
Don
-----Original Message-----
From: ActiveDir-owner@mail.activedir.org
[mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Brian Desmond
Sent: Monday, November 03, 2008 9:06 AM
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] Indexing
I'd index them and be done with it. Exchange 2007 setup indexes the
extension attributes btw.
Thanks,
Brian Desmond
brian@briandesmond.com
c - 312.731.3132
-----Original Message-----
From: ActiveDir-owner@mail.activedir.org
[mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Matt Casey
Sent: Monday, November 03, 2008 10:30 AM
To: ActiveDir@mail.activedir.org
Subject: [ActiveDir] Indexing
Hi All,
I am looking for feedback on indexing additional attributes in AD to
support an increased volume of LDAP queries. I am currently working
on several projects, including automated group management, that will
be leveraging data stored in user and group objects for various other
automated tasks. Some examples of attributes I currently use are;
employeeID, title, department, departmentNumber, employeeType, and a
few Exchange extensionAttributes.
What are some common justifications to index or not to index? It
would also be interesting to know what issues people have run into
when indexing various attributes in AD. The dit in my environment is
not that large, ~500MB, and space it not really a concern at this
point. To my knowledge we run only the default indexes for
AD(win2k3)/Exchange(2k3)/possibly Cisco(unity).
Thanks in advance!
-matt
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
| | | |
| mcasey
Posts:75
| | 11/03/2008 7:37 PM |
| Thanks for the responses! I believe the employeeID index will
probably provide us the most benefit since its value should represent
a unique object in the directory. Other values such as our job codes
and department codes should benefit as well, there appears to be a few
hundred distinct values stored in AD. Based on Don H's feedback I may
leave employeeType alone because there appears to only be 6 different
values for that field.
How you do think I should treat values that only have around 30 unique
values in the directory? For example, 4000 users divided among 30
specialization codes. Would the query processor leverage the
additional index(not as unique as employeeID, not as generic as
employeeType)? I will do some testing in a test domain but as usual
it may be hard to see how things will behave in production.
Dmitri, I inherited an OU structure that makes it difficult to target
queries effectively anywhere other than domain root. I will
definitely your comments in mind as we work on cleaning up our
structure.
On Mon, Nov 3, 2008 at 3:45 PM, Dmitri Gavrilov
<dmitrig@exchange.microsoft.com> wrote:
> One other thing to keep in mind is that regular indices are global, i.e. they index all the data in the DIT.
> Thus, they are most useful when you do global searches (e.g. find a person in the domain or forest by his employeeID).
> But if you run OU-scoped searches, then the query processor may choose not to use the global index, and resort to walking the OU hierarchy (ancestry index) instead.
>
> Again, as Don recommends, do try to introduce the indices, and see if they are actually being utilized. It's best to do it on real (or close enough) data, and with real (or close enough) queries.
>
> Dmitri
>
> -----Original Message-----
> From: ActiveDir-owner@mail.activedir.org [mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Don Hacherl
> Sent: Monday, November 03, 2008 11:50 AM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] Indexing
>
> I'd recommend indexing them and then NOT being done with it. Instead, turn
> up the logging level for searching (as shown in
> http://msdn.microsoft.com/en-us/library/ms808539.aspx), run your queries,
> and then at the results to see which of your newly added indices were
> actually used satisfying your queries.
>
> You're likely to find that indices on attributes with lots of unique values
> (e.g., employeeID) are actually used, while indices on attributes with only
> a few possible values (e.g., employeeType if that's just "full time" vs
> "part time") are considered by the query processor but never used. You want
> to keep the indices that the QP actually uses, and delete indices that are
> not used.
>
> Don
>
> -----Original Message-----
> From: ActiveDir-owner@mail.activedir.org
> [mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Brian Desmond
> Sent: Monday, November 03, 2008 9:06 AM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] Indexing
>
> I'd index them and be done with it. Exchange 2007 setup indexes the
> extension attributes btw.
>
> Thanks,
> Brian Desmond
> brian@briandesmond.com
>
> c - 312.731.3132
>
>
> -----Original Message-----
> From: ActiveDir-owner@mail.activedir.org
> [mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Matt Casey
> Sent: Monday, November 03, 2008 10:30 AM
> To: ActiveDir@mail.activedir.org
> Subject: [ActiveDir] Indexing
>
> Hi All,
> I am looking for feedback on indexing additional attributes in AD to
> support an increased volume of LDAP queries. I am currently working
> on several projects, including automated group management, that will
> be leveraging data stored in user and group objects for various other
> automated tasks. Some examples of attributes I currently use are;
> employeeID, title, department, departmentNumber, employeeType, and a
> few Exchange extensionAttributes.
>
> What are some common justifications to index or not to index? It
> would also be interesting to know what issues people have run into
> when indexing various attributes in AD. The dit in my environment is
> not that large, ~500MB, and space it not really a concern at this
> point. To my knowledge we run only the default indexes for
> AD(win2k3)/Exchange(2k3)/possibly Cisco(unity).
>
> Thanks in advance!
>
> -matt
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.activedir.org/ma/default.aspx
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.activedir.org/ma/default.aspx
>
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.activedir.org/ma/default.aspx
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.activedir.org/ma/default.aspx
>
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
| | | |
| DonH
Posts:65
| | 11/03/2008 8:20 PM |
| The QP generally uses one index per query (ignoring complicated queries),
and will use whichever index most tightly encloses the result set. How this
works with your queries depends on your data and your queries. Again I
refer you to the "Tracking Index Use" section of
http://msdn.microsoft.com/en-us/library/ms808539.aspx.
If you're running a query of the form
(&(employeeID=1234)(employeeType=Temp)) then the employeeID index will
always be the one chosen, and the presence of the employeeType index will
only slow things down a little (by causing the QP to evaluate another plan
that turns out never to be optimal).
Really, the only way to do it is load the data on a live system, run the
queries, and look at the event log QP results. If you don't want to muck
with your production system (coward!) then I personally recommend ADAM for
this. You can sync the user data over using some tool whose name I never
could remember, and then do all the schema mucking and QP testing on a
private ADAM system. This both keeps you from doing anything untoward in
production and makes your test results much easier to read.
Don
-----Original Message-----
From: ActiveDir-owner@mail.activedir.org
[mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Matt Casey
Sent: Monday, November 03, 2008 4:35 PM
To: ActiveDir@mail.activedir.org
Subject: Re: [ActiveDir] Indexing
Thanks for the responses! I believe the employeeID index will
probably provide us the most benefit since its value should represent
a unique object in the directory. Other values such as our job codes
and department codes should benefit as well, there appears to be a few
hundred distinct values stored in AD. Based on Don H's feedback I may
leave employeeType alone because there appears to only be 6 different
values for that field.
How you do think I should treat values that only have around 30 unique
values in the directory? For example, 4000 users divided among 30
specialization codes. Would the query processor leverage the
additional index(not as unique as employeeID, not as generic as
employeeType)? I will do some testing in a test domain but as usual
it may be hard to see how things will behave in production.
Dmitri, I inherited an OU structure that makes it difficult to target
queries effectively anywhere other than domain root. I will
definitely your comments in mind as we work on cleaning up our
structure.
On Mon, Nov 3, 2008 at 3:45 PM, Dmitri Gavrilov
<dmitrig@exchange.microsoft.com> wrote:
> One other thing to keep in mind is that regular indices are global, i.e.
they index all the data in the DIT.
> Thus, they are most useful when you do global searches (e.g. find a person
in the domain or forest by his employeeID).
> But if you run OU-scoped searches, then the query processor may choose not
to use the global index, and resort to walking the OU hierarchy (ancestry
index) instead.
>
> Again, as Don recommends, do try to introduce the indices, and see if they
are actually being utilized. It's best to do it on real (or close enough)
data, and with real (or close enough) queries.
>
> Dmitri
>
> -----Original Message-----
> From: ActiveDir-owner@mail.activedir.org
[mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Don Hacherl
> Sent: Monday, November 03, 2008 11:50 AM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] Indexing
>
> I'd recommend indexing them and then NOT being done with it. Instead,
turn
> up the logging level for searching (as shown in
> http://msdn.microsoft.com/en-us/library/ms808539.aspx), run your queries,
> and then at the results to see which of your newly added indices were
> actually used satisfying your queries.
>
> You're likely to find that indices on attributes with lots of unique
values
> (e.g., employeeID) are actually used, while indices on attributes with
only
> a few possible values (e.g., employeeType if that's just "full time" vs
> "part time") are considered by the query processor but never used. You
want
> to keep the indices that the QP actually uses, and delete indices that are
> not used.
>
> Don
>
> -----Original Message-----
> From: ActiveDir-owner@mail.activedir.org
> [mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Brian Desmond
> Sent: Monday, November 03, 2008 9:06 AM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] Indexing
>
> I'd index them and be done with it. Exchange 2007 setup indexes the
> extension attributes btw.
>
> Thanks,
> Brian Desmond
> brian@briandesmond.com
>
> c - 312.731.3132
>
>
> -----Original Message-----
> From: ActiveDir-owner@mail.activedir.org
> [mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Matt Casey
> Sent: Monday, November 03, 2008 10:30 AM
> To: ActiveDir@mail.activedir.org
> Subject: [ActiveDir] Indexing
>
> Hi All,
> I am looking for feedback on indexing additional attributes in AD to
> support an increased volume of LDAP queries. I am currently working
> on several projects, including automated group management, that will
> be leveraging data stored in user and group objects for various other
> automated tasks. Some examples of attributes I currently use are;
> employeeID, title, department, departmentNumber, employeeType, and a
> few Exchange extensionAttributes.
>
> What are some common justifications to index or not to index? It
> would also be interesting to know what issues people have run into
> when indexing various attributes in AD. The dit in my environment is
> not that large, ~500MB, and space it not really a concern at this
> point. To my knowledge we run only the default indexes for
> AD(win2k3)/Exchange(2k3)/possibly Cisco(unity).
>
> Thanks in advance!
>
> -matt
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.activedir.org/ma/default.aspx
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.activedir.org/ma/default.aspx
>
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.activedir.org/ma/default.aspx
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.activedir.org/ma/default.aspx
>
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
| | | |
| neilruston
Posts:164
| | 11/04/2008 4:16 AM |
| In addition to the arguments given already:
When indexing attributes, I try to ascertain how the searches will be
performed, which then determine if and how those attributes will/should
be indexed.
The schema supports 3 types of index:
- Index over object (searchFlags bit 0 set to 1)
- Index object in each container (searchFlags bits 1 and 0 set to 1)
- Tuple / medial searches / index (searchFlags bits 5 and 0 set to 1)
The attributes in question may be queried in different ways, thus
leading to those attributes all being indexed, but in slightly different
ways (as per the above).
Further reading here:
http://technet.microsoft.com/en-us/library/cc755809.aspx
I'd also agree with earlier comments regarding the assessment of whether
it is appropriate and beneficial to index attributes.
Thanks,
neil
-----Original Message-----
From: ActiveDir-owner@mail.activedir.org
[mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Don Hacherl
Sent: 04 November 2008 01:16
To: ActiveDir@mail.activedir.org
Subject: RE: [ActiveDir] Indexing
The QP generally uses one index per query (ignoring complicated
queries),
and will use whichever index most tightly encloses the result set. How
this
works with your queries depends on your data and your queries. Again I
refer you to the "Tracking Index Use" section of
http://msdn.microsoft.com/en-us/library/ms808539.aspx.
If you're running a query of the form
(&(employeeID=1234)(employeeType=Temp)) then the employeeID index will
always be the one chosen, and the presence of the employeeType index
will
only slow things down a little (by causing the QP to evaluate another
plan
that turns out never to be optimal).
Really, the only way to do it is load the data on a live system, run the
queries, and look at the event log QP results. If you don't want to
muck
with your production system (coward!) then I personally recommend ADAM
for
this. You can sync the user data over using some tool whose name I
never
could remember, and then do all the schema mucking and QP testing on a
private ADAM system. This both keeps you from doing anything untoward
in
production and makes your test results much easier to read.
Don
-----Original Message-----
From: ActiveDir-owner@mail.activedir.org
[mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Matt Casey
Sent: Monday, November 03, 2008 4:35 PM
To: ActiveDir@mail.activedir.org
Subject: Re: [ActiveDir] Indexing
Thanks for the responses! I believe the employeeID index will
probably provide us the most benefit since its value should represent
a unique object in the directory. Other values such as our job codes
and department codes should benefit as well, there appears to be a few
hundred distinct values stored in AD. Based on Don H's feedback I may
leave employeeType alone because there appears to only be 6 different
values for that field.
How you do think I should treat values that only have around 30 unique
values in the directory? For example, 4000 users divided among 30
specialization codes. Would the query processor leverage the
additional index(not as unique as employeeID, not as generic as
employeeType)? I will do some testing in a test domain but as usual
it may be hard to see how things will behave in production.
Dmitri, I inherited an OU structure that makes it difficult to target
queries effectively anywhere other than domain root. I will
definitely your comments in mind as we work on cleaning up our
structure.
On Mon, Nov 3, 2008 at 3:45 PM, Dmitri Gavrilov
<dmitrig@exchange.microsoft.com> wrote:
> One other thing to keep in mind is that regular indices are global,
i.e.
they index all the data in the DIT.
> Thus, they are most useful when you do global searches (e.g. find a
person
in the domain or forest by his employeeID).
> But if you run OU-scoped searches, then the query processor may choose
not
to use the global index, and resort to walking the OU hierarchy
(ancestry
index) instead.
>
> Again, as Don recommends, do try to introduce the indices, and see if
they
are actually being utilized. It's best to do it on real (or close
enough)
data, and with real (or close enough) queries.
>
> Dmitri
>
> -----Original Message-----
> From: ActiveDir-owner@mail.activedir.org
[mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Don Hacherl
> Sent: Monday, November 03, 2008 11:50 AM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] Indexing
>
> I'd recommend indexing them and then NOT being done with it. Instead,
turn
> up the logging level for searching (as shown in
> http://msdn.microsoft.com/en-us/library/ms808539.aspx), run your
queries,
> and then at the results to see which of your newly added indices were
> actually used satisfying your queries.
>
> You're likely to find that indices on attributes with lots of unique
values
> (e.g., employeeID) are actually used, while indices on attributes with
only
> a few possible values (e.g., employeeType if that's just "full time"
vs
> "part time") are considered by the query processor but never used.
You
want
> to keep the indices that the QP actually uses, and delete indices that
are
> not used.
>
> Don
>
> -----Original Message-----
> From: ActiveDir-owner@mail.activedir.org
> [mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Brian Desmond
> Sent: Monday, November 03, 2008 9:06 AM
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] Indexing
>
> I'd index them and be done with it. Exchange 2007 setup indexes the
> extension attributes btw.
>
> Thanks,
> Brian Desmond
> brian@briandesmond.com
>
> c - 312.731.3132
>
>
> -----Original Message-----
> From: ActiveDir-owner@mail.activedir.org
> [mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Matt Casey
> Sent: Monday, November 03, 2008 10:30 AM
> To: ActiveDir@mail.activedir.org
> Subject: [ActiveDir] Indexing
>
> Hi All,
> I am looking for feedback on indexing additional attributes in AD to
> support an increased volume of LDAP queries. I am currently working
> on several projects, including automated group management, that will
> be leveraging data stored in user and group objects for various other
> automated tasks. Some examples of attributes I currently use are;
> employeeID, title, department, departmentNumber, employeeType, and a
> few Exchange extensionAttributes.
>
> What are some common justifications to index or not to index? It
> would also be interesting to know what issues people have run into
> when indexing various attributes in AD. The dit in my environment is
> not that large, ~500MB, and space it not really a concern at this
> point. To my knowledge we run only the default indexes for
> AD(win2k3)/Exchange(2k3)/possibly Cisco(unity).
>
> Thanks in advance!
>
> -matt
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.activedir.org/ma/default.aspx
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.activedir.org/ma/default.aspx
>
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.activedir.org/ma/default.aspx
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.activedir.org/ma/default.aspx
>
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
Barclays Wealth is the wealth management division of Barclays Bank PLC. This email may relate to or be sent from other members of the Barclays Group.
The availability of products and services may be limited by the applicable laws and regulations in certain jurisdictions. The Barclays Group does not normally accept or offer business instructions via internet email. Any action that you might take upon this message might be at your own risk.
This email and any attachments are confidential and intended solely for the addressee and may also be privileged or exempt from disclosure under applicable law. If you are not the addressee, or have received this email in error, please notify the sender immediately, delete it from your system and do not copy, disclose or otherwise act upon any part of this email or its attachments.
Internet communications are not guaranteed to be secure or without viruses. The Barclays Group does not accept responsibility for any loss arising from unauthorised access to, or interference with, any Internet communications by any third party, or from the transmission of any viruses. Replies to this email may be monitored by the Barclays Group for operational or business reasons.
Any opinion or other information in this email or its attachments that does not relate to the business of the Barclays Group is personal to the sender and is not given or endorsed by the Barclays Group.
Barclays Bank PLC. Registered in England and Wales (registered no. 1026167).
Registered Office: 1 Churchill Place, London, E14 5HP, United Kingdom.
Barclays Bank PLC is authorised and regulated by the Financial Services Authority.
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
| | | |
| mcasey
Posts:75
| | 11/04/2008 8:12 AM |
| Thanks again. I read through the articles you guys mentioned, I
appreciate the info. I must admit that 'coward' sometimes comes to
mind when i think about the change management review that comes before
I make the change. Actually making the change in production is a
little less of a concern 
In the meantime I will get another instance of ADAM going and see how
much fun I can have with that. Thanks again!
On Tue, Nov 4, 2008 at 4:11 AM, <neil.ruston@barclayswealth.com> wrote:
> In addition to the arguments given already:
>
> When indexing attributes, I try to ascertain how the searches will be
> performed, which then determine if and how those attributes will/should
> be indexed.
>
> The schema supports 3 types of index:
> - Index over object (searchFlags bit 0 set to 1)
> - Index object in each container (searchFlags bits 1 and 0 set to 1)
> - Tuple / medial searches / index (searchFlags bits 5 and 0 set to 1)
>
> The attributes in question may be queried in different ways, thus
> leading to those attributes all being indexed, but in slightly different
> ways (as per the above).
>
> Further reading here:
> http://technet.microsoft.com/en-us/library/cc755809.aspx
>
> I'd also agree with earlier comments regarding the assessment of whether
> it is appropriate and beneficial to index attributes.
>
>
>
> Thanks,
> neil
>
>
> -----Original Message-----
> From: ActiveDir-owner@mail.activedir.org
> [mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Don Hacherl
> Sent: 04 November 2008 01:16
> To: ActiveDir@mail.activedir.org
> Subject: RE: [ActiveDir] Indexing
>
> The QP generally uses one index per query (ignoring complicated
> queries),
> and will use whichever index most tightly encloses the result set. How
> this
> works with your queries depends on your data and your queries. Again I
> refer you to the "Tracking Index Use" section of
> http://msdn.microsoft.com/en-us/library/ms808539.aspx.
>
> If you're running a query of the form
> (&(employeeID=1234)(employeeType=Temp)) then the employeeID index will
> always be the one chosen, and the presence of the employeeType index
> will
> only slow things down a little (by causing the QP to evaluate another
> plan
> that turns out never to be optimal).
>
> Really, the only way to do it is load the data on a live system, run the
> queries, and look at the event log QP results. If you don't want to
> muck
> with your production system (coward!) then I personally recommend ADAM
> for
> this. You can sync the user data over using some tool whose name I
> never
> could remember, and then do all the schema mucking and QP testing on a
> private ADAM system. This both keeps you from doing anything untoward
> in
> production and makes your test results much easier to read.
>
> Don
>
> -----Original Message-----
> From: ActiveDir-owner@mail.activedir.org
> [mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Matt Casey
> Sent: Monday, November 03, 2008 4:35 PM
> To: ActiveDir@mail.activedir.org
> Subject: Re: [ActiveDir] Indexing
>
> Thanks for the responses! I believe the employeeID index will
> probably provide us the most benefit since its value should represent
> a unique object in the directory. Other values such as our job codes
> and department codes should benefit as well, there appears to be a few
> hundred distinct values stored in AD. Based on Don H's feedback I may
> leave employeeType alone because there appears to only be 6 different
> values for that field.
>
> How you do think I should treat values that only have around 30 unique
> values in the directory? For example, 4000 users divided among 30
> specialization codes. Would the query processor leverage the
> additional index(not as unique as employeeID, not as generic as
> employeeType)? I will do some testing in a test domain but as usual
> it may be hard to see how things will behave in production.
>
> Dmitri, I inherited an OU structure that makes it difficult to target
> queries effectively anywhere other than domain root. I will
> definitely your comments in mind as we work on cleaning up our
> structure.
>
>
>
> On Mon, Nov 3, 2008 at 3:45 PM, Dmitri Gavrilov
> <dmitrig@exchange.microsoft.com> wrote:
>> One other thing to keep in mind is that regular indices are global,
> i.e.
> they index all the data in the DIT.
>> Thus, they are most useful when you do global searches (e.g. find a
> person
> in the domain or forest by his employeeID).
>> But if you run OU-scoped searches, then the query processor may choose
> not
> to use the global index, and resort to walking the OU hierarchy
> (ancestry
> index) instead.
>>
>> Again, as Don recommends, do try to introduce the indices, and see if
> they
> are actually being utilized. It's best to do it on real (or close
> enough)
> data, and with real (or close enough) queries.
>>
>> Dmitri
>>
>> -----Original Message-----
>> From: ActiveDir-owner@mail.activedir.org
> [mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Don Hacherl
>> Sent: Monday, November 03, 2008 11:50 AM
>> To: ActiveDir@mail.activedir.org
>> Subject: RE: [ActiveDir] Indexing
>>
>> I'd recommend indexing them and then NOT being done with it. Instead,
> turn
>> up the logging level for searching (as shown in
>> http://msdn.microsoft.com/en-us/library/ms808539.aspx), run your
> queries,
>> and then at the results to see which of your newly added indices were
>> actually used satisfying your queries.
>>
>> You're likely to find that indices on attributes with lots of unique
> values
>> (e.g., employeeID) are actually used, while indices on attributes with
> only
>> a few possible values (e.g., employeeType if that's just "full time"
> vs
>> "part time") are considered by the query processor but never used.
> You
> want
>> to keep the indices that the QP actually uses, and delete indices that
> are
>> not used.
>>
>> Don
>>
>> -----Original Message-----
>> From: ActiveDir-owner@mail.activedir.org
>> [mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Brian Desmond
>> Sent: Monday, November 03, 2008 9:06 AM
>> To: ActiveDir@mail.activedir.org
>> Subject: RE: [ActiveDir] Indexing
>>
>> I'd index them and be done with it. Exchange 2007 setup indexes the
>> extension attributes btw.
>>
>> Thanks,
>> Brian Desmond
>> brian@briandesmond.com
>>
>> c - 312.731.3132
>>
>>
>> -----Original Message-----
>> From: ActiveDir-owner@mail.activedir.org
>> [mailto:ActiveDir-owner@mail.activedir.org] On Behalf Of Matt Casey
>> Sent: Monday, November 03, 2008 10:30 AM
>> To: ActiveDir@mail.activedir.org
>> Subject: [ActiveDir] Indexing
>>
>> Hi All,
>> I am looking for feedback on indexing additional attributes in AD to
>> support an increased volume of LDAP queries. I am currently working
>> on several projects, including automated group management, that will
>> be leveraging data stored in user and group objects for various other
>> automated tasks. Some examples of attributes I currently use are;
>> employeeID, title, department, departmentNumber, employeeType, and a
>> few Exchange extensionAttributes.
>>
>> What are some common justifications to index or not to index? It
>> would also be interesting to know what issues people have run into
>> when indexing various attributes in AD. The dit in my environment is
>> not that large, ~500MB, and space it not really a concern at this
>> point. To my knowledge we run only the default indexes for
>> AD(win2k3)/Exchange(2k3)/possibly Cisco(unity).
>>
>> Thanks in advance!
>>
>> -matt
>> List info : http://www.activedir.org/List.aspx
>> List FAQ : http://www.activedir.org/ListFAQ.aspx
>> List archive: http://www.activedir.org/ma/default.aspx
>> List info : http://www.activedir.org/List.aspx
>> List FAQ : http://www.activedir.org/ListFAQ.aspx
>> List archive: http://www.activedir.org/ma/default.aspx
>>
>> List info : http://www.activedir.org/List.aspx
>> List FAQ : http://www.activedir.org/ListFAQ.aspx
>> List archive: http://www.activedir.org/ma/default.aspx
>> List info : http://www.activedir.org/List.aspx
>> List FAQ : http://www.activedir.org/ListFAQ.aspx
>> List archive: http://www.activedir.org/ma/default.aspx
>>
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.activedir.org/ma/default.aspx
>
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.activedir.org/ma/default.aspx
> Barclays Wealth is the wealth management division of Barclays Bank PLC. This email may relate to or be sent from other members of the Barclays Group.
>
> The availability of products and services may be limited by the applicable laws and regulations in certain jurisdictions. The Barclays Group does not normally accept or offer business instructions via internet email. Any action that you might take upon this message might be at your own risk.
>
> This email and any attachments are confidential and intended solely for the addressee and may also be privileged or exempt from disclosure under applicable law. If you are not the addressee, or have received this email in error, please notify the sender immediately, delete it from your system and do not copy, disclose or otherwise act upon any part of this email or its attachments.
>
> Internet communications are not guaranteed to be secure or without viruses. The Barclays Group does not accept responsibility for any loss arising from unauthorised access to, or interference with, any Internet communications by any third party, or from the transmission of any viruses. Replies to this email may be monitored by the Barclays Group for operational or business reasons.
>
> Any opinion or other information in this email or its attachments that does not relate to the business of the Barclays Group is personal to the sender and is not given or endorsed by the Barclays Group.
>
> Barclays Bank PLC. Registered in England and Wales (registered no. 1026167).
> Registered Office: 1 Churchill Place, London, E14 5HP, United Kingdom.
>
> Barclays Bank PLC is authorised and regulated by the Financial Services Authority.
> List info : http://www.activedir.org/List.aspx
> List FAQ : http://www.activedir.org/ListFAQ.aspx
> List archive: http://www.activedir.org/ma/default.aspx
>
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
| | | |
| joe
Posts:106
| | 11/04/2008 9:23 AM |
| The other thing you can do that is helpful is to use a tool like ldp.exe or
adfind that supports the stats control. This allows you to execute your
searches with the tool and get feedback on index usage and other query
processor stats returned directly to the client.
The stats control does require admin access on the server to use though.
Joe K.
----- Original Message -----
From: "Matt Casey" <mcasey726@gmail.com>
To: <ActiveDir@mail.activedir.org>
Sent: Tuesday, November 04, 2008 7:07 AM
Subject: Re: [ActiveDir] Indexing
> Thanks again. I read through the articles you guys mentioned, I
> appreciate the info. I must admit that 'coward' sometimes comes to
> mind when i think about the change management review that comes before
> I make the change. Actually making the change in production is a
> little less of a concern
>
> In the meantime I will get another instance of ADAM going and see how
> much fun I can have with that. Thanks again!
>
List info : http://www.activedir.org/List.aspx
List FAQ : http://www.activedir.org/ListFAQ.aspx
List archive: http://www.activedir.org/ma/default.aspx
| | | |
|
|