From 7299b3561fa579b6b90bbf3becc821b91429c9ac Mon Sep 17 00:00:00 2001

From: Noriko Hosoi <nhosoi@redhat.com>

Date: Fri, 9 May 2014 09:53:36 -0700

Subject: [PATCH 209/225] Ticket #47750 - Creating a glue fails if one above

 level is a conflict or missing

Including the fix for Ticket #47696 - Large Searches Hang - Possibly entryrdn related

Bug Descriptions:

There were multiple issues to break the parent-child relationship when a

resurrect occurred.

When resurrecting a tombstone, there is a possibility that an entry having

the same name was already added.  In the case, the code consdered the entry

is already there, which confused the numsubordinates and tombstone-num-

subordinates count.  If it is detected, it creates a conflict glue entry

instead of a pure glue entry.  The 2 numsubordinates counts were also not

correctly updated in the resurrection case. When the entryrdn was introduced,

Slapi_RDN was held in the entry, which was not correctly updated in the

resurrect case.

Fix Descriptions:

[replication/urp.c]

urp_fixup_rename_entry sets the parent uniqueid if it is a RESURRECT mode

(called from tombstone_to_glue) as being done in urp_fixup_add_entry, which

is necessary to retrieve the correct parent in ldbm_back_modrdn.

[replication/urp_glue.c]

In create_glue_entry,

. there was a possibility to pass a tombstone DN (nsuniqueid=...,<DN>) is

  used to create a glue entry.  If found, the part is removed before passing

  it to create a glue entry.

. there is a possibility to create a glue entry fails due to the parent entry

  not found.  This occurs when the parent entry has turned to be a conflict

  entry.  In the case, instead of retrying the glue creation, it logs an

  error and returns.

[replication/urp_tombstone.c]

In tombstone_to_glue,

. turning a tombstone to an ordinary glue entry could fail with the entry

  already exists.  (Note: ldbm_back_add guarantees that the found entry has

  a different nsuniqueid from this tombstone has.)  This situation occurs an

  entry having the same dn could have been added/replicated by this moment.

  And the entry cannot be used for this tombstone to resurrect since the

  subordinates/tombstonesubordinates relationships get corrupted. To maintain

  the relationship properly, the tombstone is resurrected as a conflict, glue

  entry using urp_fixup_rename_entry.  If it still returns LDAP_ALREADY_EXISTS,

  it is assured the expected glue entry exists since the same nsuniqueid is

  shared.

[slapd/back-ldbm/ldbm_add.c]

In ldbm_back_add,

. in is_resurect_operation case, a SLAPI_ADD_EXISTING_DN_ENTRY entry and a

  SLAPI_ADD_EXISTING_UNIQUEID_ENTRY entry are identical, it returns SUCCESS

  instead of LDAP_ALREADY_EXISTS.

. in is_resurect_operation case, the DN is modified from the tombstone DN

  to the ordinary DN.  The DN is stored in e_sdn as well as in e_srdn in the

  Slapi_Entry.  The e_srdn part was not updated.  This patch fixes it and

  e_sdn and e_srdn are in sync.

. parent_update_on_childchange adjusts the count of numsubordinates and

  tombstonenumsubordinates.  It only took PARENTUPDATE_ADD|_DELETE operations.

  This patch adds PARENTUPDATE_RESURECT operation to increment numsubordinates

  and decrement tombstonenumsubordinates at the same time.

. in is_resurect_operation case, a tombstone to be replaced with a glue entry

  needs to be deleted from the entryrdn index, which was missing.

[slapd/back-ldbm/ldbm_delete.c]

. Check the target DN if it starts with "nsuniqueid=...,", if it is, log an

  error and returns.

[slapd/back-ldbm/ldbm_modrdn.c]

. If it is is_resurect_operation case, it allows to rename a tombstone entry.

  This is used to support to change a tombstone entry to a conflict glue entry

  in tombstone_to_glue.

. ldbm_back_modify checks if the renamed entry has children or not and if

  there are, they are updated to point to the renamed entry.  The check

  function slapi_entry_has_children was just checking the numsubordinates

  attribute type.  If a positive value is returned, it took care all the

  children including the tombstones, but if the renamed entry had tombstoned

  children only, it considered there were no children and the tombstoned

  children were not updated to point the renamed entry.  This patch fixes it.

[slapd/back-ldbm/ldbm_entryrdn.c]

. Added a flag to entryrdn_rename_subtree and entryrdn_get_subordinates to

  treat the given DN a tombstoned DN if it is.

In addition, helper APIs such as slapi_rdn_is_multivalued and slapi_rdn_is_

conflict, and slapi_is_special_rdn are added.

https://fedorahosted.org/389/ticket/47750

Reviewed by mreynolds@redhat.com (Thank you, Mark!!)

(cherry picked from commit 9e5b3332cc33167837fce0477c8a4d0edc42f0db)

---

 ldap/servers/plugins/replication/repl5.h         |   1 +

 ldap/servers/plugins/replication/urp.c           |  79 +++--

 ldap/servers/plugins/replication/urp.h           |   5 +-

 ldap/servers/plugins/replication/urp_glue.c      |  32 +-

 ldap/servers/plugins/replication/urp_tombstone.c |  55 +++-

 ldap/servers/slapd/add.c                         |  15 +-

 ldap/servers/slapd/back-ldbm/back-ldbm.h         |   3 +-

 ldap/servers/slapd/back-ldbm/backentry.c         |   1 +

 ldap/servers/slapd/back-ldbm/cache.c             | 103 ++++---

 ldap/servers/slapd/back-ldbm/ldbm_add.c          | 362 ++++++++++++++---------

 ldap/servers/slapd/back-ldbm/ldbm_delete.c       | 337 ++++++++++++++-------

 ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c     |  12 +-

 ldap/servers/slapd/back-ldbm/ldbm_modify.c       |  41 ++-

 ldap/servers/slapd/back-ldbm/ldbm_modrdn.c       | 281 ++++++++++++++----

 ldap/servers/slapd/back-ldbm/ldbm_search.c       |   2 +-

 ldap/servers/slapd/back-ldbm/ldif2ldbm.c         |   2 +-

 ldap/servers/slapd/back-ldbm/parents.c           |  31 +-

 ldap/servers/slapd/back-ldbm/proto-back-ldbm.h   |   5 +-

 ldap/servers/slapd/entry.c                       |  38 ++-

 ldap/servers/slapd/rdn.c                         | 108 ++++++-

 ldap/servers/slapd/slapi-plugin.h                |  45 +++

 ldap/servers/slapd/slapi-private.h               |   7 +

 ldap/servers/slapd/util.c                        |  67 +++++

 23 files changed, 1184 insertions(+), 448 deletions(-)

diff --git a/ldap/servers/plugins/replication/repl5.h b/ldap/servers/plugins/replication/repl5.h

index b10b358..a5761be 100644

--- a/ldap/servers/plugins/replication/repl5.h

+++ b/ldap/servers/plugins/replication/repl5.h

@@ -60,6 +60,7 @@

 #include "llist.h"

 #include "repl5_ruv.h"

 #include "cl4.h"

+#include "plstr.h"

 #define REPLICA_TYPE_WINDOWS 1

 #define REPLICA_TYPE_MULTIMASTER 0

diff --git a/ldap/servers/plugins/replication/urp.c b/ldap/servers/plugins/replication/urp.c

index 9787ff7..0182af3 100644

--- a/ldap/servers/plugins/replication/urp.c

+++ b/ldap/servers/plugins/replication/urp.c

@@ -58,7 +58,6 @@ static int urp_naming_conflict_removal (Slapi_PBlock *pb, char *sessionid, CSN *

 static int mod_namingconflict_attr (const char *uniqueid, const Slapi_DN *entrysdn, const Slapi_DN *conflictsdn, CSN *opcsn);

 static int del_replconflict_attr (Slapi_Entry *entry, CSN *opcsn, int opflags);

 static char *get_dn_plus_uniqueid(char *sessionid,const Slapi_DN *oldsdn,const char *uniqueid);

-static char *get_rdn_plus_uniqueid(char *sessionid,const char *olddn,const char *uniqueid);

 static int is_suffix_entry (Slapi_PBlock *pb, Slapi_Entry *entry, Slapi_DN **parenddn);

 /*

@@ -372,6 +371,9 @@ urp_modrdn_operation( Slapi_PBlock *pb )

 			slapi_entry_get_sdn (target_entry), "renameTombstone", opcsn);

 		*/

 		op_result = LDAP_NO_SUCH_OBJECT;

+		slapi_log_error(SLAPI_LOG_REPL, sessionid,

+		                "urp_modrdn: target_entry %s is a tombstone; returning NO_SUCH_OBJECT.\n",

+		                slapi_entry_get_dn(target_entry));

 		slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result);

 		if (op_result == 0)

 		{

@@ -448,16 +450,16 @@ urp_modrdn_operation( Slapi_PBlock *pb )

 				slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result);

 				rc= -1; /* Ignore this Operation */

 				PROFILE_POINT; /* ModRDN Conflict; Entry with Target DN Exists;

-								  Unique ID already in RDN - Change to Lost and Found entry */

+				                  Unique ID already in RDN - Change to Lost and Found entry */

 				goto bailout;

 			}

 			mod_namingconflict_attr (op_uniqueid, target_sdn, existing_sdn, opcsn);

 			slapi_pblock_set(pb, SLAPI_MODRDN_NEWRDN, newrdn_with_uniqueid); 

 			slapi_log_error(slapi_log_urp, sessionid,

-					"Naming conflict MODRDN. Rename target entry to %s\n",

-					newrdn_with_uniqueid );

+			                "urp_modrdn_operation: Naming conflict MODRDN. Rename target entry from %s to %s\n",

+			                newrdn, newrdn_with_uniqueid );

-			rc= slapi_setbit_int(rc,SLAPI_RTN_BIT_FETCH_EXISTING_DN_ENTRY);

+			rc= slapi_setbit_int(rc, SLAPI_RTN_BIT_FETCH_EXISTING_DN_ENTRY);

 			PROFILE_POINT; /* ModRDN Conflict; Entry with Target DN Exists; Rename Operation Entry */

 			goto bailout;

 		}

@@ -608,14 +610,20 @@ urp_delete_operation( Slapi_PBlock *pb )

 		op_result= LDAP_NO_SUCH_OBJECT;

 		slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result);

 		rc= -1; /* Don't apply the Delete */

+		slapi_log_error(slapi_log_urp, "no sessionid",

+		                "Entry %s does not exist; returning NO_SUCH_OBJECT.\n",

+		                slapi_entry_get_dn(deleteentry));

 		PROFILE_POINT; /* Delete Operation; Entry not exist. */

 	}

 	else if(is_tombstone_entry(deleteentry))

 	{

 		/* The entry is already a Tombstone, ignore this delete. */

-		op_result= LDAP_SUCCESS;

+		op_result= LDAP_ALREADY_EXISTS;

 		slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result);

 		rc = -1; /* Don't apply the Delete */

+		slapi_log_error(slapi_log_urp, "no sessionid",

+		                "Entry %s is already a tombstone; returning ALREADY_EXISTS.\n",

+		                slapi_entry_get_dn(deleteentry));

 		PROFILE_POINT; /* Delete Operation; Already a Tombstone. */

 	}

 	else /* The entry to be deleted exists and is not a tombstone */

@@ -639,7 +647,10 @@ urp_delete_operation( Slapi_PBlock *pb )

 			op_result= LDAP_SUCCESS;

 			slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result);

 			rc = -1; /* Don't apply the Delete */

-		    PROFILE_POINT; /* Delete Operation; Entry has children. */

+			slapi_log_error(slapi_log_urp, sessionid,

+			                "Entry %s was turned to a glue; returning SUCCESS.\n",

+			                slapi_entry_get_dn(deleteentry) );

+			PROFILE_POINT; /* Delete Operation; Entry has children. */

 		}

 	}

 	return rc;

@@ -785,7 +796,7 @@ urp_fixup_add_entry (Slapi_Entry *e, const char *target_uniqueid, const char *pa

 }

 int

-urp_fixup_rename_entry (Slapi_Entry *entry, const char *newrdn, int opflags)

+urp_fixup_rename_entry (Slapi_Entry *entry, const char *newrdn, const char *parentuniqueid, int opflags)

 {

 	Slapi_PBlock *newpb;

     Slapi_Operation *op;

@@ -813,7 +824,12 @@ urp_fixup_rename_entry (Slapi_Entry *entry, const char *newrdn, int opflags)

 	opcsn = (CSN *)entry_get_dncsn (entry);

     slapi_pblock_get (newpb, SLAPI_OPERATION, &op);

     operation_set_csn (op, opcsn);

-

+	if (parentuniqueid)

+	{

+		struct slapi_operation_parameters *op_params;

+		slapi_pblock_get( newpb, SLAPI_OPERATION_PARAMETERS, &op_params );

+		op_params->p.p_modrdn.modrdn_newsuperior_address.uniqueid = (char*)parentuniqueid; /* Consumes parentuniqueid */

+	}

 	slapi_modrdn_internal_pb(newpb); 

     slapi_pblock_get(newpb, SLAPI_PLUGIN_INTOP_RESULT, &op_result);

@@ -1012,7 +1028,7 @@ urp_annotate_dn (char *sessionid, Slapi_Entry *entry, CSN *opcsn, const char *op

 	if(newrdn!=NULL)

 	{

 		mod_namingconflict_attr (uniqueid, basesdn, basesdn, opcsn);

-		op_result = urp_fixup_rename_entry ( entry, newrdn, 0 );

+		op_result = urp_fixup_rename_entry ( entry, newrdn, NULL, 0 );

 		switch(op_result)

 		{

 		case LDAP_SUCCESS:

@@ -1191,7 +1207,7 @@ urp_naming_conflict_removal ( Slapi_PBlock *pb, char *sessionid, CSN *opcsn, con

 	 * is done after DB lock was released. The backend modrdn

 	 * will acquire the DB lock if it sees this flag.

 	 */

-	op_result = urp_fixup_rename_entry (min_naming_conflict_entry, newrdnstr, OP_FLAG_ACTION_INVOKE_FOR_REPLOP);

+	op_result = urp_fixup_rename_entry (min_naming_conflict_entry, newrdnstr, NULL, OP_FLAG_ACTION_INVOKE_FOR_REPLOP);

 	if ( op_result != LDAP_SUCCESS )

 	{

 	    slapi_log_error (slapi_log_urp, sessionid,

@@ -1224,15 +1240,19 @@ bailout:

 static char *

 get_dn_plus_uniqueid(char *sessionid, const Slapi_DN *oldsdn, const char *uniqueid)

 {

-	Slapi_RDN *rdn= slapi_rdn_new();

-	char *newdn;

+	Slapi_RDN *rdn = slapi_rdn_new();

+	char *newdn = NULL;

+	int rc = slapi_rdn_init_all_sdn_ext(rdn, oldsdn, 1);

+	if (rc) {

+		slapi_log_error(SLAPI_LOG_FATAL, sessionid,

+		                "Failed to convert %s to RDN\n", slapi_sdn_get_dn(oldsdn));

+		goto bail;

+	}

 	PR_ASSERT(uniqueid!=NULL);

 	/* Check if the RDN already contains the Unique ID */

-	slapi_rdn_set_dn(rdn, slapi_sdn_get_dn(oldsdn));

-	if(slapi_rdn_contains(rdn,SLAPI_ATTR_UNIQUEID,uniqueid,strlen(uniqueid)))

-	{

+	if (slapi_rdn_is_conflict(rdn)) {

 		/* The Unique ID is already in the RDN.

 		 * This is a highly improbable collision.

 		 * It suggests that a duplicate UUID was generated.

@@ -1241,7 +1261,6 @@ get_dn_plus_uniqueid(char *sessionid, const Slapi_DN *oldsdn, const char *unique

 		 */

 		slapi_log_error(SLAPI_LOG_FATAL, sessionid,

 				"Annotated DN %s has naming conflict\n", slapi_sdn_get_dn(oldsdn) );

-		newdn= NULL;

 	}

 	else

 	{

@@ -1250,21 +1269,27 @@ get_dn_plus_uniqueid(char *sessionid, const Slapi_DN *oldsdn, const char *unique

 		newdn = slapi_ch_smprintf("%s,%s", slapi_rdn_get_rdn(rdn), parentdn);

 		slapi_ch_free_string(&parentdn);

 	}

+bail:

 	slapi_rdn_free(&rdn;;

 	return newdn;

 }

-static char *

+char *

 get_rdn_plus_uniqueid(char *sessionid, const char *olddn, const char *uniqueid)

 {

-	char *newrdn;

+	char *newrdn = NULL;

 	/* Check if the RDN already contains the Unique ID */

-	Slapi_DN *sdn= slapi_sdn_new_dn_byval(olddn);

-	Slapi_RDN *rdn= slapi_rdn_new();

-	slapi_sdn_get_rdn(sdn,rdn);

+	Slapi_DN *sdn = slapi_sdn_new_dn_byval(olddn);

+	Slapi_RDN *rdn = slapi_rdn_new();

+	int rc = slapi_rdn_init_all_sdn_ext(rdn, sdn, 1);

+	if (rc) {

+		slapi_log_error(SLAPI_LOG_FATAL, sessionid,

+		                "Failed to convert %s to RDN\n", olddn);

+		goto bail;

+	}

+

 	PR_ASSERT(uniqueid!=NULL);

-	if(slapi_rdn_contains(rdn,SLAPI_ATTR_UNIQUEID,uniqueid,strlen(uniqueid)))

-	{

+	if (slapi_rdn_is_conflict(rdn)) {

 		/* The Unique ID is already in the RDN.

 		 * This is a highly improbable collision.

 		 * It suggests that a duplicate UUID was generated.

@@ -1272,14 +1297,14 @@ get_rdn_plus_uniqueid(char *sessionid, const char *olddn, const char *uniqueid)

 		 * require admin intercession

 		 */

 		slapi_log_error(SLAPI_LOG_FATAL, sessionid,

-				"Annotated DN %s has naming conflict\n", olddn );

-		newrdn= NULL;

+		                "Annotated RDN %s has naming conflict\n", olddn);

 	}

 	else

 	{

 		slapi_rdn_add(rdn,SLAPI_ATTR_UNIQUEID,uniqueid);

-		newrdn= slapi_ch_strdup(slapi_rdn_get_rdn(rdn));

+		newrdn = slapi_ch_strdup(slapi_rdn_get_rdn(rdn));

 	}

+bail:

 	slapi_sdn_free(&sdn;;

 	slapi_rdn_free(&rdn;;

 	return newrdn;

diff --git a/ldap/servers/plugins/replication/urp.h b/ldap/servers/plugins/replication/urp.h

index ec7c94d..c7d8ca5 100644

--- a/ldap/servers/plugins/replication/urp.h

+++ b/ldap/servers/plugins/replication/urp.h

@@ -55,11 +55,12 @@ int urp_delete_operation( Slapi_PBlock *pb );

 int urp_post_delete_operation( Slapi_PBlock *pb );

 int urp_modrdn_operation( Slapi_PBlock *pb );

 int urp_post_modrdn_operation( Slapi_PBlock *pb );

+char *get_rdn_plus_uniqueid(char *sessionid, const char *olddn, const char *uniqueid);

 /* urp internal ops */

 int urp_fixup_add_entry (Slapi_Entry *e, const char *target_uniqueid, const char *parentuniqueid, CSN *opcsn, int opflags);

 int urp_fixup_delete_entry (const char *uniqueid, const char *dn, CSN *opcsn, int opflags);

-int urp_fixup_rename_entry (Slapi_Entry *entry, const char *newrdn, int opflags);

+int urp_fixup_rename_entry (Slapi_Entry *entry, const char *newrdn, const char *parentuniqueid, int opflags);

 int urp_fixup_modify_entry (const char *uniqueid, const Slapi_DN *sdn, CSN *opcsn, Slapi_Mods *smods, int opflags);

 int is_suffix_dn (Slapi_PBlock *pb, const Slapi_DN *dn, Slapi_DN **parenddn);

@@ -78,6 +79,6 @@ PRBool get_glue_csn(const Slapi_Entry *entry, const CSN **gluecsn);

  * urp_tombstone.c

  */

 int is_tombstone_entry(const Slapi_Entry* entry);

-int tombstone_to_glue(Slapi_PBlock *pb, const char *sessionid, Slapi_Entry *entry, const Slapi_DN *parentdn, const char *reason, CSN *opcsn);

+int tombstone_to_glue(Slapi_PBlock *pb, char *sessionid, Slapi_Entry *entry, const Slapi_DN *parentdn, const char *reason, CSN *opcsn);

 int entry_to_tombstone ( Slapi_PBlock *pb, Slapi_Entry *entry );

 PRBool get_tombstone_csn(const Slapi_Entry *entry, const CSN **delcsn);

diff --git a/ldap/servers/plugins/replication/urp_glue.c b/ldap/servers/plugins/replication/urp_glue.c

index 6eeb826..199d4de 100644

--- a/ldap/servers/plugins/replication/urp_glue.c

+++ b/ldap/servers/plugins/replication/urp_glue.c

@@ -153,7 +153,7 @@ static const char *glue_entry =

 	"objectclass: top\n"

 	"objectclass: extensibleObject\n" /* JCMREPL - To avoid schema checking. */

 	"objectclass: glue\n"

-	"nsuniqueid: %s\n"

+	"nsuniqueid: %s\n" /* this uniqueid is set to Slapi_Entry in slapi_str2entry. */

 	"%s: %s\n"; /* Add why it's been created */

 static int

@@ -166,15 +166,15 @@ do_create_glue_entry(const Slapi_RDN *rdn, const Slapi_DN *superiordn, const cha

 	Slapi_DN *sdn = NULL;

 	Slapi_RDN *newrdn = slapi_rdn_new_rdn(rdn);

 	char *estr, *rdnstr, *rdntype, *rdnval, *rdnpair;

-	sdn = slapi_sdn_new_dn_byval(slapi_sdn_get_ndn(superiordn));

+	sdn = slapi_sdn_new_ndn_byval(slapi_sdn_get_ndn(superiordn));

 	slapi_sdn_add_rdn(sdn,rdn);

 	/* must take care of multi-valued rdn: split rdn into different lines introducing

 	 * '\n' between each type/value pair. 

-	 */	

+	 */

 	alloc_len = RDNBUFSIZE;

 	rdnstr = slapi_ch_malloc(alloc_len);

-	rdnpair = rdnstr;	

+	rdnpair = rdnstr;

 	*rdnpair = '\0';   /* so that strlen(rdnstr) may return 0 the first time it's called */

 	while ((rdnval_index = slapi_rdn_get_next(newrdn, rdnval_index, &rdntype, &rdnval)) != -1) {

 	        rdntype_len = strlen(rdntype);

@@ -188,7 +188,7 @@ do_create_glue_entry(const Slapi_RDN *rdn, const Slapi_DN *superiordn, const cha

 		}

 		slapi_ldif_put_type_and_value_with_options(&rdnpair, rdntype, rdnval, rdnval_len, LDIF_OPT_NOWRAP);

 		*rdnpair = '\0';

-	}	  

+	}

 	estr= slapi_ch_smprintf(glue_entry, slapi_sdn_get_ndn(sdn), rdnstr, uniqueid, 

 			ATTR_NSDS5_REPLCONFLICT, reason);

 	slapi_ch_free((void**)&rdnstr);

@@ -196,9 +196,7 @@ do_create_glue_entry(const Slapi_RDN *rdn, const Slapi_DN *superiordn, const cha

 	slapi_ch_free((void**)&newrdn);

 	e = slapi_str2entry( estr, 0 );

 	PR_ASSERT(e!=NULL);

-	if ( e!=NULL )

-	{

-		slapi_entry_set_uniqueid (e, slapi_ch_strdup(uniqueid));

+	if (e) {

 		op_result = urp_fixup_add_entry (e, NULL, NULL, opcsn, 0);

 	}

 	slapi_ch_free_string(&estr);

@@ -232,7 +230,7 @@ create_glue_entry ( Slapi_PBlock *pb, char *sessionid, Slapi_DN *dn, const char

 		slapi_pblock_get( pb, SLAPI_BACKEND, &backend );

 		slapi_sdn_get_backend_parent ( dn, superiordn, backend );

-		slapi_sdn_get_rdn ( dn, rdn );

+		slapi_rdn_set_dn_ext(rdn, slapi_sdn_get_dn(dn), 1/* skip nsuniqeid=..., in dn */);

 		while(!done)

 		{

@@ -246,16 +244,30 @@ create_glue_entry ( Slapi_PBlock *pb, char *sessionid, Slapi_DN *dn, const char

 					done= 1;

 					break;

 				case LDAP_ALREADY_EXISTS:

+				{

+					struct slapi_operation_parameters *op_params;

+					/* This is okay.  While creating a glue, a real entry was added. */

 					slapi_log_error ( SLAPI_LOG_FATAL, repl_plugin_name,

 						"%s: Skipped creating glue entry %s uniqueid=%s reason Entry Already Exists\n",

 						sessionid, dnstr, uniqueid);

+					op_result = LDAP_SUCCESS;

+					/* If we could not create a glue having the nsuniqueid,

+					 * we have to abandon it. */

+					slapi_pblock_get(pb, SLAPI_OPERATION_PARAMETERS, &op_params);

+					slapi_ch_free_string(&op_params->p.p_add.parentuniqueid);

 					done= 1;

 					break;

+				}

 				case LDAP_NO_SUCH_OBJECT:

 					/* The parent is missing */

 					{

 					/* JCMREPL - Create the parent ... recursion?... but what's the uniqueid? */

-					PR_ASSERT(0); /* JCMREPL */

+					slapi_log_error (SLAPI_LOG_FATAL, repl_plugin_name,

+						"%s: Can't created glue entry %s uniqueid=%s, error %d; "

+						"Possibly, parent entry is a conflict entry.\n",

+						sessionid, dnstr, uniqueid, op_result);

+					done= 1;

+					break;

 					}

 				default:

 					slapi_log_error ( SLAPI_LOG_FATAL, repl_plugin_name,

diff --git a/ldap/servers/plugins/replication/urp_tombstone.c b/ldap/servers/plugins/replication/urp_tombstone.c

index a61f1e5..68cb269 100644

--- a/ldap/servers/plugins/replication/urp_tombstone.c

+++ b/ldap/servers/plugins/replication/urp_tombstone.c

@@ -84,7 +84,7 @@ get_tombstone_csn(const Slapi_Entry *entry, const CSN **delcsn)

 static int

 tombstone_to_glue_resolve_parent (

 	Slapi_PBlock *pb,

-	const char *sessionid,

+	char *sessionid,

 	const Slapi_DN *parentdn,

 	const char *parentuniqueid,

 	CSN *opcsn)

@@ -144,7 +144,7 @@ tombstone_to_glue_resolve_parent (

 int

 tombstone_to_glue (

 	Slapi_PBlock *pb,

-	const char *sessionid,

+	char *sessionid,

 	Slapi_Entry *tombstoneentry,

 	const Slapi_DN *tombstonedn,

 	const char *reason,

@@ -156,6 +156,7 @@ tombstone_to_glue (

 	Slapi_Entry *addingentry;

 	const char *addingdn;

 	int op_result;

+	int rdn_is_conflict = 0;

 	/* JCMREPL

 	 * Nothing logged to the 5.0 Change Log

@@ -170,18 +171,19 @@ tombstone_to_glue (

 	 * which won't help us identify the correct backend to search.

 	 */

 	is_suffix_dn_ext (pb, tombstonedn, &parentdn, 1 /* is_tombstone */);

-	parentuniqueid= slapi_entry_attr_get_charptr (tombstoneentry,

-			SLAPI_ATTR_VALUE_PARENT_UNIQUEID); /* Allocated */

+	parentuniqueid= slapi_entry_attr_get_charptr (tombstoneentry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); /* Allocated */

 	tombstone_to_glue_resolve_parent (pb, sessionid, parentdn, parentuniqueid, opcsn);

-	slapi_sdn_free(&parentdn);

-    /* Submit an Add operation to turn the tombstone entry into glue. */

+	/* Submit an Add operation to turn the tombstone entry into glue. */

 	/*

 	 * The tombstone is stored with an invalid DN, we must fix this.

 	 */

 	addingentry = slapi_entry_dup(tombstoneentry);

 	addingdn = slapi_sdn_get_dn(tombstonedn);

 	slapi_entry_set_sdn(addingentry, tombstonedn);

+	/* not just e_sdn, e_rsdn needs to be updated. */

+	slapi_rdn_set_all_dn(slapi_entry_get_srdn(addingentry), slapi_entry_get_dn_const(addingentry));

+	rdn_is_conflict = slapi_rdn_is_conflict(slapi_entry_get_srdn(addingentry));

 	if (!slapi_entry_attr_hasvalue(addingentry, ATTR_NSDS5_REPLCONFLICT, reason))

 	{

@@ -189,18 +191,57 @@ tombstone_to_glue (

 		slapi_entry_add_string(addingentry, ATTR_NSDS5_REPLCONFLICT, reason);

 	}

 	tombstoneuniqueid= slapi_entry_get_uniqueid(tombstoneentry);

-	op_result = urp_fixup_add_entry (addingentry, tombstoneuniqueid, parentuniqueid, opcsn, OP_FLAG_RESURECT_ENTRY);

+	/* 

+	 * addingentry and parentuniqueid are consumed in urp_fixup_add_entry,

+	 * regardless of the result.

+	 * Note: addingentry is not really consumed in ldbm_back_add.

+	 * tombstoneentry from DB/entry cache is duplicated and turned to be a glue.

+	 * This addingentry is freed in op_shared_add.

+	 */

+	op_result = urp_fixup_add_entry (addingentry, tombstoneuniqueid, slapi_ch_strdup(parentuniqueid), opcsn, OP_FLAG_RESURECT_ENTRY);

+	if ((LDAP_ALREADY_EXISTS == op_result) && !rdn_is_conflict) {

+		/* conflict -- there's already the same named entry added.

+		 * But this to-be-glued entry needs to be added since this is a parent of child entries...

+		 * So, rename this tombstone parententry a conflict, glue entry.

+		 * Instead of "fixup_add", we have to "fixup_rename"...

+		 * */

+		char *conflictrdn = get_rdn_plus_uniqueid(sessionid, addingdn, tombstoneuniqueid);

+		if (conflictrdn) {

+			addingentry = slapi_entry_dup(tombstoneentry);

+			if (!slapi_entry_attr_hasvalue(addingentry, ATTR_NSDS5_REPLCONFLICT, reason)) {

+				/* Add the reason of turning it to glue - The backend code will use it*/

+				slapi_entry_add_string(addingentry, ATTR_NSDS5_REPLCONFLICT, reason);

+			}

+			slapi_log_error (SLAPI_LOG_FATAL, repl_plugin_name,

+			                 "%s: Can't resurrect tombstone to glue reason '%s'. "

+			                 "Try with conflict dn %s, error=%d\n",

+			                 sessionid, reason, addingdn, op_result);

+			op_result = urp_fixup_rename_entry(addingentry, (const char *)conflictrdn, parentuniqueid,

+			                                   OP_FLAG_RESURECT_ENTRY|OP_FLAG_TOMBSTONE_ENTRY);

+			slapi_entry_free(addingentry);

+			addingentry = NULL;

+		}

+	}

+	slapi_ch_free_string(&parentuniqueid);

 	if (op_result == LDAP_SUCCESS)

 	{

 		slapi_log_error (slapi_log_urp, repl_plugin_name,

 			"%s: Resurrected tombstone %s to glue reason '%s'\n", sessionid, addingdn, reason);

 	}

+	else if (LDAP_ALREADY_EXISTS == op_result)

+	{

+		slapi_log_error(slapi_log_urp, repl_plugin_name,

+		                "%s: No need to turn tombstone %s to glue; it was already resurrected.\n",

+		                sessionid, addingdn);

+		op_result = LDAP_SUCCESS;

+	}

 	else

 	{

 		slapi_log_error (SLAPI_LOG_FATAL, repl_plugin_name,

 			"%s: Can't resurrect tombstone %s to glue reason '%s', error=%d\n",

 			sessionid, addingdn, reason, op_result);

 	}

+	slapi_sdn_free(&parentdn);

 	return op_result;

 }

diff --git a/ldap/servers/slapd/add.c b/ldap/servers/slapd/add.c

index 44a9aa2..1ce975a 100644

--- a/ldap/servers/slapd/add.c

+++ b/ldap/servers/slapd/add.c

@@ -699,10 +699,17 @@ static void op_shared_add (Slapi_PBlock *pb)

 				slapi_pblock_get(pb, SLAPI_ENTRY_POST_OP, &pse;;

 				do_ps_service(pse, NULL, LDAP_CHANGETYPE_ADD, 0);

-

-				/* If be_add succeeded, then e is consumed.  We

-				 * set e to NULL to prevent freeing it ourselves. */

-				e = NULL;

+				/* 

+				 * If be_add succeeded, then e is consumed except the resurect case.

+				 * If it is resurect, the corresponding tombstone entry is resurected

+				 * and put into the cache.

+				 * Otherwise, we set e to NULL to prevent freeing it ourselves.

+				 */

+				if (operation_is_flag_set(operation,OP_FLAG_RESURECT_ENTRY) && save_e) {

+					e = save_e;

+				} else {

+					e = NULL;

+				}

 			}

 			else

 			{

diff --git a/ldap/servers/slapd/back-ldbm/back-ldbm.h b/ldap/servers/slapd/back-ldbm/back-ldbm.h

index 8ad3c20..d27a664 100644

--- a/ldap/servers/slapd/back-ldbm/back-ldbm.h

+++ b/ldap/servers/slapd/back-ldbm/back-ldbm.h

@@ -893,7 +893,8 @@ typedef struct _back_search_result_set

 /* operation for parent_update_on_childchange */

 #define PARENTUPDATE_ADD       0x1

 #define PARENTUPDATE_DEL       0x2

-#define PARENTUPDATE_MASK      (PARENTUPDATE_ADD|PARENTUPDATE_DEL)

+#define PARENTUPDATE_RESURECT  0x4

+#define PARENTUPDATE_MASK      (PARENTUPDATE_ADD|PARENTUPDATE_DEL|PARENTUPDATE_RESURECT)

 #define PARENTUPDATE_CREATE_TOMBSTONE  0x10

 #define PARENTUPDATE_DELETE_TOMBSTONE  0x20

diff --git a/ldap/servers/slapd/back-ldbm/backentry.c b/ldap/servers/slapd/back-ldbm/backentry.c

index 52468ad..114c2df 100644

--- a/ldap/servers/slapd/back-ldbm/backentry.c

+++ b/ldap/servers/slapd/back-ldbm/backentry.c

@@ -52,6 +52,7 @@ backentry_free( struct backentry **bep )

 		return;

 	}

 	ep = *bep;

+	PR_ASSERT(ep->ep_state & (ENTRY_STATE_DELETED|ENTRY_STATE_NOTINCACHE));

 	if ( ep->ep_entry != NULL ) {

 		slapi_entry_free( ep->ep_entry );

 	}

diff --git a/ldap/servers/slapd/back-ldbm/cache.c b/ldap/servers/slapd/back-ldbm/cache.c

index 865f1ef..5510a9a 100644

--- a/ldap/servers/slapd/back-ldbm/cache.c

+++ b/ldap/servers/slapd/back-ldbm/cache.c

@@ -890,7 +890,7 @@ entrycache_remove_int(struct cache *cache, struct backentry *e)

         }

         else

         {

-            LOG("remove %d from id hash failed\n", e->ep_id, 0, 0);

+            LOG("remove %s (%d) from id hash failed\n", ndn, e->ep_id, 0);

         }

     }

 #ifdef UUIDCACHE_ON 

@@ -984,13 +984,19 @@ int cache_replace(struct cache *cache, void *oldptr, void *newptr)

 static int entrycache_replace(struct cache *cache, struct backentry *olde,

                               struct backentry *newe)

 {

-    int found;

+    int found = 0;

+    int found_in_dn = 0;

+    int found_in_id = 0;

+#ifdef UUIDCACHE_ON

+    int found_in_uuid = 0;

+#endif

     const char *oldndn;

     const char *newndn;

 #ifdef UUIDCACHE_ON 

     const char *olduuid;

     const char *newuuid;

 #endif

+    struct backentry *alte = NULL;

     LOG("=> entrycache_replace (%s) -> (%s)\n", backentry_get_ndn(olde),

         backentry_get_ndn(newe), 0);

@@ -1013,31 +1019,21 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde,

      * cache tables, operation error 

      */

     if ( (olde->ep_state & ENTRY_STATE_NOTINCACHE) == 0 ) {

-        int found_in_dn = remove_hash(cache->c_dntable, (void *)oldndn, strlen(oldndn));

-        int found_in_id = remove_hash(cache->c_idtable, &(olde->ep_id), sizeof(ID));

+        found_in_dn = remove_hash(cache->c_dntable, (void *)oldndn, strlen(oldndn));

+        found_in_id = remove_hash(cache->c_idtable, &(olde->ep_id), sizeof(ID));

 #ifdef UUIDCACHE_ON

-        int found_in_uuid = remove_hash(cache->c_uuidtable, (void *)olduuid, strlen(olduuid));

+        found_in_uuid = remove_hash(cache->c_uuidtable, (void *)olduuid, strlen(olduuid));

 #endif

         found = found_in_dn && found_in_id;

 #ifdef UUIDCACHE_ON

         found = found && found_in_uuid;

 #endif

-        if (!found) {

-#ifdef UUIDCACHE_ON

-            LOG("entry cache replace: cache index tables out of sync - found dn [%d] id [%d] uuid [%d]\n",

-                found_in_dn, found_in_id, found_in_uuid);

-#else

-            LOG("entry cache replace: cache index tables out of sync - found dn [%d] id [%d]\n",

-                found_in_dn, found_in_id, 0);

-#endif

-            PR_Unlock(cache->c_mutex);

-            return 1;

-        }

     }

-    if (! entry_same_dn(newe, (void *)oldndn) &&

-         (newe->ep_state & ENTRY_STATE_NOTINCACHE) == 0) {

-        /* if we're doing a modrdn, the new entry can be in the dn table

-         * already, so we need to remove that too.

+    /* If fails, we have to make sure the both entires are removed from the cache,

+     * otherwise, we have no idea what's left in the cache or not... */

+    if (!entry_same_dn(newe, (void *)oldndn) && (newe->ep_state & ENTRY_STATE_NOTINCACHE) == 0) {

+        /* if we're doing a modrdn or turning an entry to a tombstone,

+         * the new entry can be in the dn table already, so we need to remove that too.

          */

         if (remove_hash(cache->c_dntable, (void *)newndn, strlen(newndn)))

         {

@@ -1046,20 +1042,44 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde,

             LOG("entry cache replace remove entry size %lu\n", newe->ep_size, 0, 0);

         }

     }

-

+    /* 

+     * The old entry could have been "removed" between the add and this replace,

+     * The entry is NOT freed, but NOT in the dn hash.

+     * which could happen since the entry is not necessarily locked.

+     * This is ok.

+     */

+    olde->ep_state = ENTRY_STATE_DELETED; /* olde is removed from the cache, so set DELETED here. */

+    if (!found) {

+        if (olde->ep_state & ENTRY_STATE_DELETED) {

+            LOG("entry cache replace (%s): cache index tables out of sync - found dn [%d] id [%d]; but the entry is alreay deleted.\n",

+                oldndn, found_in_dn, found_in_id);

+        } else {

+#ifdef UUIDCACHE_ON

+            LOG("entry cache replace: cache index tables out of sync - found dn [%d] id [%d] uuid [%d]\n",

+                found_in_dn, found_in_id, found_in_uuid);

+#else

+            LOG("entry cache replace (%s): cache index tables out of sync - found dn [%d] id [%d]\n",

+                oldndn, found_in_dn, found_in_id);

+#endif

+            PR_Unlock(cache->c_mutex);

+            return 1;

+        }