码迷,mamicode.com
首页 > 其他好文 > 详细

lucene indexwriter锁机制

时间:2014-07-01 09:40:53      阅读:295      评论:0      收藏:0      [点我收藏+]

标签:lucene indexwriter lock

最近有个项目要用solr,solr是基于lucene的,今天在测试indexwriter时遇到了lock的问题:

测试代码:

import java.io.File;
import java.io.IOException;
import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.FSDirectory;
import org.apache.lucene.util.Version;
public class TestLock {
    private Directory dir;
    public static TestLock ttt;
    public static IndexWriter writer2;
    private Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_48);
    private IndexWriterConfig iwc = new IndexWriterConfig(Version.LUCENE_48, analyzer);
    public void init() throws Exception {
        String pathFile = "D://luceneindex";
        try{
               dir = FSDirectory. open(new File(pathFile));
        } catch (IOException e) {        
            throw new RuntimeException(e);
        }
        IndexWriter writer = getWriter();             
        System. out.println("init ok,test IndexWriter lock" );
        LockTest( writer);      
        //writer.close();
    }
    public IndexWriter getWriter() throws Exception {
        //analyzer.close();
        return new IndexWriter(dir, iwc);
    }
    public void LockTest(IndexWriter w1) {
       try{
               if(w1.isLocked (dir )){
                     System. out.println("write1 locked" );
                     IndexWriterConfig iwc1 = new IndexWriterConfig(Version.LUCENE_48 , analyzer );
                      writer2 = new IndexWriter(dir, iwc1); 
              } else{
                     System. out.println("write1 not locked" );                      
              }      
       } catch(Exception e){
              e.printStackTrace();
       }       
    }  
    public static void main(String[] args){
       ttt = new TestLock();
       try{
               ttt.init();
       } catch(Exception e){
              e. printStackTrace();              
       }      
    }
}

报错信息:

org.apache.lucene.store.LockObtainFailedException : Lock obtain timed out: NativeFSLock@D:\luceneindex\write.lock: java.nio.channels.OverlappingFileLockException
       at org.apache.lucene.store.Lock.obtain( Lock.java:89)
       at org.apache.lucene.index.IndexWriter.<init>( IndexWriter.java:710)
       at TestLock.LockTest( TestLock.java:38)
       at TestLock.init( TestLock.java:26)
       at TestLock.main( TestLock.java:51)
Caused by: java.nio.channels.OverlappingFileLockException
       at sun.nio.ch.SharedFileLockTable.checkList(Unknown Source)
       at sun.nio.ch.SharedFileLockTable.add(Unknown Source)
       at sun.nio.ch.FileChannelImpl.tryLock(Unknown Source)
       at java.nio.channels.FileChannel.tryLock(Unknown Source)
       at org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:148)
       at org.apache.lucene.store.Lock.obtain( Lock.java:100)
       ... 4 more

从错误信息可以看到是IndexWriter获取锁出错导致,从堆栈信息可以看到,是在运行IndexWriter的构造方法时,涉及到锁的操作。

查看IndexWriter的相关源码:

doc:
Opening an IndexWriter creates a lock file for the directory in use. Trying to open another IndexWriter  on the same directory will lead to a
LockObtainFailedException. The  LockObtainFailedException is also thrown if an IndexReader on the same directory is used to delete documents
from the index.

两个和锁相关的属性:

  public static final String WRITE_LOCK_NAME = "write.lock" ;
  private Lock writeLock;

在IndexWriter的构造函数中:

  public IndexWriter(Directory d, IndexWriterConfig conf) throws IOException
....
     directory = d;
.....
    writeLock = directory.makeLock(WRITE_LOCK_NAME);
    if (! writeLock.obtain(config .getWriteLockTimeout())) // obtain write lock(尝试获取锁)
      throw new LockObtainFailedException("Index locked for write: " + writeLock );

其中Lock是一个抽象类(org.apache.lucene.store.Lock):   

锁的获取主要实现是在obtain方法:

  public static long LOCK_POLL_INTERVAL = 1000;
  public static final long LOCK_OBTAIN_WAIT_FOREVER = -1;
  public final boolean obtain(long lockWaitTimeout) throws IOException {
    failureReason = null;
    boolean locked = obtain();
    if (lockWaitTimeout < 0 && lockWaitTimeout != LOCK_OBTAIN_WAIT_FOREVER )
      throw new IllegalArgumentException("lockWaitTimeout should be LOCK_OBTAIN_WAIT_FOREVER or a non-negative number (got " + lockWaitTimeout + ")");
    long maxSleepCount = lockWaitTimeout / LOCK_POLL_INTERVAL;
    long sleepCount = 0;
    while (!locked) {
      if (lockWaitTimeout != LOCK_OBTAIN_WAIT_FOREVER && sleepCount++ >= maxSleepCount) {
        String reason = "Lock obtain timed out: " + this .toString();
        if (failureReason != null) {
          reason += ": " + failureReason ;
        }
        LockObtainFailedException e = new LockObtainFailedException(reason);
        if (failureReason != null) {
          e.initCause( failureReason);
        }
        throw e;
      }
      try {
        Thread. sleep(LOCK_POLL_INTERVAL);
      } catch (InterruptedException ie) {
        throw new ThreadInterruptedException(ie);
      }
      locked = obtain();
    }
    return locked;
  }

可以看到有由下面几个因素决定:

1.lockWaitTimeout 超时时间(总时间,超过这个时间即timeout),默认1000ms

2.LOCK_OBTAIN_WAIT_FOREVER  是否无限获取(-1),如果设置为-1,会永不超生

3.LOCK_POLL_INTERVAL  重试间隔时间,默认1000ms

在关闭IndexWriter时调用close方法(实现了Closeable接口的类都会有close方法)即可正常释放锁

更改代码为如下即可:

  public void LockTest(IndexWriter w1) {
       try{
               if(w1.isLocked (dir )){
                     System. out.println("write1 locked" );
                     w1.close();
                     IndexWriterConfig iwc1 = new IndexWriterConfig(Version.LUCENE_48 , analyzer );
                      writer2 = new IndexWriter(dir, iwc1); 
              } else{
                     System. out.println("write1 not locked" );                      
              }      
       } catch(Exception e){
              e.printStackTrace();
       }       
    }

另外关于lock的判断和unlock方法如下:

 public static boolean isLocked(Directory directory) throws IOException {  // 判断写目录是否被lock
    return directory.makeLock( WRITE_LOCK_NAME).isLocked();
  }
  /**
   * Forcibly unlocks the index in the named directory.
   * <P>
   * Caution: this should only be used by failure recovery code,
   * when it is known that no other process nor thread is in fact
   * currently accessing this index.
   */
  public static void unlock(Directory directory) throws IOException { // 解锁
    directory.makeLock(IndexWriter. WRITE_LOCK_NAME).close();
  }

本文出自 “菜光光的博客” 博客,请务必保留此出处http://caiguangguang.blog.51cto.com/1652935/1432795

lucene indexwriter锁机制,布布扣,bubuko.com

lucene indexwriter锁机制

标签:lucene indexwriter lock

原文地址:http://caiguangguang.blog.51cto.com/1652935/1432795

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!