[Yanel-dev] problem re-starting Yanel containing a realm with search lock

[Josias Thöny]

Michael Wechner wrote:
> Josias Thöny wrote:
> 
>> Michael Wechner wrote:
>>
>>> Hi
>>>
>>> I have recently noticed the following problem
>>>
>>> 21420 2008-05-19 11:28:24,992 [http-9290-Processor25] ERROR 
>>> org.wyona.yanel.core.map.RealmManager.readRealms():259  - Error 
>>> setting up realm [yanel-website]: 
>>> /home/ulysses/src/realms/yanel-website/realm.xml: 
>>> java.lang.Exception: Could not configure realm [yanel-website] with 
>>> config file: /home/ulysses/src/realms/yanel-website/realm.xml: 
>>> org.wyona.yarep.core.RepositoryException: Could not create 
>>> repository: yanel-website 
>>> /home/ulysses/src/realms/yanel-website/config/res-config-repository.xml 
>>> Could not read repository configuration: Lock obtain timed out: 
>>> SimpleFSLock@/home/ulysses/src/realms/yanel-website/res-configs-repo/search-index/write.lock 
>>>
>>>        at org.wyona.yanel.core.map.Realm.<init>(Realm.java:104)
>>>        at 
>>> org.wyona.yanel.core.map.RealmManager.readRealms(RealmManager.java:232)
>>>        at 
>>> org.wyona.yanel.core.map.RealmManager.<init>(RealmManager.java:118)
>>>        at 
>>> org.wyona.yanel.core.map.RealmManager.<init>(RealmManager.java:76)
>>>        at org.wyona.yanel.core.Yanel.init(Yanel.java:80)
>>>        at 
>>> org.wyona.yanel.servlet.YanelServlet.init(YanelServlet.java:154)
>>>
>>>
>>> I haven't noticed this before. Is it possible that this is some 
>>> regression of the recent repo changes? That some IndexWriter isn't 
>>> closed properly?
>>
>>
>> There was a bug in yanel that it didn't close the res-config 
>> repository upon shutdown.
>> I fixed this on 2008-05-13 16:36:47 (r36537).
> 
> 
> thanks
> 
>> Then I noticed that other repositories (ac-identities, ac-policies, 
>> custom repos) are not closed upon shutdown either. I'm not sure how to 
>> fix this.
> 
> 
> 
> well, it would be nice to shutdown these as well, but I think the 
> problem above isn't really solved by shutting down, but rather that the 
> IndexWriter needs to be closed every time after the writing has 
> finished, because in the case of cluster with multiple nodes accessing 
> the same index we will have a real problem.
> 
> Is there any reason that the IndexWriter isn't closed?

The problem is that it's not allowed to open more than one IndexWriter 
for one search index, see [1]. So if multiple threads (within one 
cluster node or spread among several nodes) start indexing at the same 
time, there will be LockObtainFailedExceptions. That's what happened in 
our customer project.
That's why I changed the behaviour to have only one IndexWriter which is 
closed when the repository is shut down. Now I realize that this doesn't 
work in a cluster. Doh!
Maybe there should be a special clustered yarep implementation which 
performs some synchronization. I think it should not be required for all 
yarep implementations to be clusterable, because it can add a lot of 
complexity and adds a performance penalty.

Josias

[1] 
http://lucene.apache.org/java/2_2_0/api/org/apache/lucene/index/IndexWriter.html

> 
>> Maybe with a finalize() method of the IdentityManager/PolicyManager?
>>
>> BTW, why is indexing enabled for the res-config repository of the 
>> yanel-website realm?
> 
> 
> no, it would make sense to disable this.
> 
> Cheers
> 
> Michi
> 
>>
>> josias
>>
>>>
>>> Also maybe 
>>> http://www.manning-sandbox.com/message.jspa?messageID=39279 helps
>>>
>>> Thanks
>>>
>>> Michi
>>>
>>
>> _______________________________________________
>> Yanel-development mailing list
>> Yanel-development at wyona.com
>> http://lists.wyona.org/cgi-bin/mailman/listinfo/yanel-development
> 
> 
>