Version 19 (modified by jgarcia@…, 5 years ago) (diff)

--

Content

  1. Ldap Backend
    1. Planning
    2. Work in progress
      1. Ldap #1
      2. Ldap #2
      3. Ldap #3
      4. Ldap #4
      5. Ldap #5
      6. Ldap #6
      7. Ldap #7
    3. Guides and stuff
      1. Ldap Object Structure
      2. Ldap directory servers tested
        1. Benchmark Results
      3. Ldap information
        1. Useful RFCs
        2. Useful Links
        3. Useful commands

Ldap Backend

Work in progress... (probably you won't be able to understand most of the contents of this page, at least nowadays... in the end the contents could make sense, i hope so...)

So, as i said before, Viewer Discretion is Advised!

The purpose of this work is to replace gconf for Ldap (exactly OpenLdap) as the storage configuration engine.

Planning

  • (1) Generation of Ldap Schemas 4,5d
    • Type of attribute types (Ldap Syntaxes) for Ebox::Types::*
      • Relationship between Ldap Type <=> Ebox::Type::xxx.yyy Ldap #1 - 1,5d
        • Ip+netmask in 2 strings or in another type?
        • Is there a type for MacAddress representation?
      • New Types? Automatic or manual generation? /client/XXXX/src/Ebox/YYYY/Types/*.pm Ldap #2 - 0,5d
    • objectclass for each model defined in a TableDescription /client/XXXX/src/Ebox/YYYY/Model/*.pm #3 - 2,5d
      • Auto generation of Ldap schema given a TableDescription (new()->table())
        • We are going to mock the behaviour, in the future we should use the ModelManager
        • objectclass with attributes namespaced
        • OIDs?!
        • First generate attributetypes, secondly (and finally) the objectclasses
  • (2) Initial Ldap Backend Implementation 9,5d
    • Replace GconfModule for LdapStuff without EBox::Module integration #4 - 3d
      • Try to use the Perl Test::Class and TDD style
    • Modify Types/* #5 - 1,5d
      • Do not know anything about store
        • Remove storeInGconf
      • Give a Hash with the values to store
      • Special case if it's new record
      • Special case if it's the ID (change RDN in LDAP)
    • Modify DataTable #6 - 2,5d
      • Find operation in another file or class
      • Do not know how to store, make Table operations, not object ones
    • Modify Row #7 - 2,5d
      • Store (create/modify) the data
      • Get info from each attribute and create a query with LdapStuff for doing the dirty work
  • More stuff to come...

Work in progress

Ldap #1

Relationship betwen ldap Type <=> Ebox::Type

  • Templates to create attributetype of some concrete SYNTAX.
    • Definition 
      AttributeTypeDescription = "(" whsp
   numericoid whsp     ; AttributeType identifier
 [ "NAME" qdescrs ]             ; name used in AttributeType
 [ "DESC" qdstring ]            ; description
 [ "OBSOLETE" whsp ]
 [ "SUP" woid ]                 ; derived from this other
                                ; AttributeType
 [ "EQUALITY" woid              ; Matching Rule name
 [ "ORDERING" woid              ; Matching Rule name
 [ "SUBSTR" woid ]              ; Matching Rule name
 [ "SYNTAX" whsp noidlen whsp ] ; Syntax OID
 [ "SINGLE-VALUE" whsp ]        ; default multi-valued
 [ "COLLECTIVE" whsp ]          ; default not collective
 [ "NO-USER-MODIFICATION" whsp ]; default user modifiable
 [ "USAGE" whsp AttributeUsage ]; default userApplications
 whsp ")"
    • Types supported
      • Boolean
      • String
      • Integer
      • Float
      • Datetime
    • Templates 
      attributetype ( __OID__ NAME '__NAME__'
 ___OPTIONS___
)

OPTIONS depends on which type we want to represent:

Boolean
  EQUALITY booleanMatch
  SYNTAX 1.3.6.1.4.1.1466.115.121.1.7

String (DirectoryString) UTF-8 encoding of ISO-10646 (Unicode)
  EQUALITY caseIgnoreMatch
  SUBSTR caseIgnoreSubstringsMatch
  ORDERING caseIgnoreOrderingMatch
  SYNTAX 1.3.6.1.4.1.1466.115.121.1.15

Integer
  EQUALITY integerMatch
  ORDERING integerOrderingMatch
  SYNTAX 1.3.6.1.4.1.1466.115.121.1.27


Float
  There is no standard for saving them, so save them as String

Datetime
    EQUALITY generalizedTimeMatch
    ORDERING generalizedTimeOrderingMatch
    SYNTAX 1.3.6.1.4.1.1466.115.121.1.24
  • Every Ebox's Types has a method (see Ldap #2 ) description() with a hash with tuples name => type
  • A function "somewhere" gets a EBox::Type (and a Module and Model name) and generates the attributetype schema definitions
    • For types that have more than one field (e.g. PortRange) the function generates more than one attributetype
    • Special cases will be:
      • Composite: do nothing
      • HasMany: No need for declare an attributetype, a hasmany field in a model, means that the foreign EBoxModel will be as a child of the current Ebox Model (the one which the HasMany attribute, but this's not concern for the schema generation)

Ldap #2

New Types? Automatic or manual generation

  • Every EBox::Type has to implement a description() method which returns a reference to a Hash with tuples name => type.
    • With this information (how the type need to be stored), we'll be able to generate schemas for every possible type created
    • e.g.: 
      package EBox::Type::Boolean;

sub description() {
    my $self = shift;
    return { 
        $self->fieldName() => EBox::Ldap::Types::Boolean 
    };
}

      package EBox::Type::Port;

sub description() {
    my $self = shift;
    return {
        'initial' => EBox::Ldap::Types::Integer,
        'end'     => EBox::Ldap::Types::Integer,
        'type'    => EBox::Ldap::Types::String
    };
}

Ldap #3

Auto generation of Ldap schema

  • For each model, firstly generate attribute types, finally generate object classes.
  • Attribute type
    • See Ldap #1 and Ldap #2
    • An attributetype depends on:
      • OID, which depends on:
        • EBox Module which the model belongs to
      • Type: this information is given by description method
      • Name, formed like "ebox$moduleName$modelName$attrName", which depends on:
        • EBox Module which the model belongs to
        • EBox Model which the attribute belongs to
  • Object class
    • Definition 
      ObjectClassDescription = "(" whsp
 numericoid whsp      ; ObjectClass identifier
 [ "NAME" qdescrs ]
 [ "DESC" qdstring ]
 [ "OBSOLETE" whsp ]
 [ "SUP" oids ]       ; Superior ObjectClasses
 [ ( "ABSTRACT" / "STRUCTURAL" / "AUXILIARY" ) whsp ]
                      ; default structural
 [ "MUST" oids ]      ; AttributeTypes
 [ "MAY" oids ]       ; AttributeTypes
  whsp ")"
    • All object class generated will be just like: 
      objectclass ( ___OID___
   NAME '___NAME___'
   DESC 'Model foo for module bar'
   SUP top
   STRUCTURAL
   MUST ( [uid|___ATTRIBUTES___] )
   MAY  ( ___ATTRIBUTES___ ) )
    • An objectclass depends on:
      • OID, which depends on:
        • EBox Module which the model belongs to
      • Name, formed like "ebox$moduleName$modelName"
      • Attributes
        • If there is some must attribute, this will be part of the RDN (the first one if there are > 1)
          • An attribute is declared as must when its unique method returns true and only has 1 field (See Ldap #2).
        • Otherwise, an uid attribute will be defined (the entries will look like: uid=1,cn=foo..., uid=2,cn=foo,.., uid=20,cn=foo...) (See Ldap Object Structure).

Ldap #4

Ldap stuff for dealing with Ldap Server

  • Reuse EBox::Ldap singleton class
    • It could be useful add methods (for schema generation) like:
      • existsSchemaFor($moduleName, $modelName)
        • Should we create schema for model $modelName and module $moduleName?
      • oidStateFor($moduleName)
        • Returns the next free oid for both attributetypes and objectclasses inside the oid for the module $moduleName (e.g. if we have defined the module Samba with 1 model and 10 attributes, this method will return {attribute => 11, object => 2 }
      • nextFreeOid()
        • Returns the next oid to define a new ebox module (each module has a concrete subtree of the ebox's oids)

Ldap #5

Ldap #6

Ldap #7

Guides and stuff

Ldap Object Structure

  • ou=ebox,$BASE_DN [here there are attr names of the object]
    • ou=modules,ou=ebox
      • ou=moduleName,ou=modules,ou=ebox
        • cn=modelName,ou=moduleName,ou=modules,ou=ebox [keys, ...]
          • uid=1,cn=modelName,ou=moduleName,ou=modules,ou=ebox
            • cn=someSubmodelName,ou=moduleName,ou=modules,ou=ebox [keys,]
              • eboxModuleNameSubmodelNameAttrName=foobarizate,cn=modelName,ou=moduleName,ou=modules,ou=ebox
              • ...
              • eboxModuleNameSubmodelNameAttrName=jate,cn=modelName,ou=moduleName,ou=modules,ou=ebox
          • uid=2,cn=modelName,ou=moduleName,ou=modules,ou=ebox
            • cn=someSubmodelName,ou=moduleName,ou=modules,ou=ebox
              • ...
          • ...
          • uid=100,cn=modelName,ou=moduleName,ou=modules,ou=ebox
        • cn=anotherModelName,ou=moduleName,ou=modules,ou=ebox [keys, ..]
          • eboxModuleNameAnotherModelNameAttrName=Foobar,cn=anotherModelName,ou=moduleName,ou=modules,ou=ebox
          • ...
          • eboxModuleNameAnotherModelNameAttrName=lalala,cn=anotherModelName,ou=moduleName,ou=modules,ou=ebox
      • ...

Ldap directory servers tested

There are some schemas and code for each one, but in brief:

  • OpenLdap (Ubuntu 8.04)
    • In the last version (the one with Ubuntu 8.10) the schema are saved in the Ldap itself.
  • Apache Directory Server
    • Same performance (in the very simple load test used...)
  • Fedora Directory Server
    • More performance (+10-20%) and some additional features but less integration.

So, for now, OpenLdap wins (but it will be almost trivial change it in the future)

Benchmark Results

# OpenLdap VS MySQL, Quad core 6600. 10k entries, 4 values
       s/iter  LDAP     DB
LDAP     3.62   --     -98%
DB   7.00e-02  5074%    --

# OpenLdap VS MySQL , Dual core E2140, 10k entries, 4 values
     s/iter  LDAP    DB
LDAP   7.29    --  -98%
DB    0.172 4151%    --

#################################################

# OpenLdap Dual core E2140, 2k entries, 104 values
       s/iter
LDAP     3.51

# OpenLdap, Dual core E2140, 2k entries, 4 values
       s/iter
LDAP     1.28

#################################################

# OpenLdap vs FedoraDS, Dual core E2140, 10k entries 4 values
          s/iter  OpenLDAP Fedora DS
OpenLDAP    6.85        --      -14%
Fedora DS   5.86       17%        --

# OpenLdap vs FedoraDS, Dual core E2140, 2k entries 104 values
          s/iter  OpenLDAP Fedora DS
OpenLDAP    3.44        --      -13%
Fedora DS   3.01       14%        --

################################################
####################  Ruby  ####################
################################################

# OpenLdap vs MySQL, Quad core 6600, 10k entries 4 values
           user     system      total        real
LDAP  10.440000   0.790000  11.230000 ( 11.566221)
DB     0.350000   0.020000   0.370000 (  0.368707)

Ldap information

We are all f*king noobs unless the opposite will be proved

Useful RFCs

Useful commands

(if you don't have knowledge of ldap... or you're as dumb as i am)

  • Remove db (be careful with the permissions!) 
    $ sudo rm /var/lib/ldap/__db.* /var/lib/ldap/*.bdb /var/lib/ldap/log.*
  • Check configuration 
    $ sudo slaptest -v
  • See all supported stuff 
    $ ldapsearch -H ldap://localhost -xLLL -s base -b "" +
  • Add a ldif file 
    $ ldapadd -H ldap://127.0.0.1:9999  -x -w passForAdmin -D 'cn=admin,dc=local' -f file_with_data.ldif
-v # verbose mode
-c # continue if fails
  • Delete stuff 
    $ ldapdelete -r -x -w passForAdmin -H ldap://localhost:9999 -D 'cn=admin,dc=local' "ou=sampleModel,cn=sample,ou=modules,ou=ebox,dc=local"
-r # recursive
  • Search stuff 
    $ ldapsearch -H ldap://127.0.0.1:9999 -xLLL -w passForAdmin -D 'cn=admin,dc=local' "(objectclass=*)" 
-E pr=3 # pagination