最近有个项目要用solr,solr是基于lucene的,今天在测试indexwriter时遇到了lock的问题:
测试代码:
import java.io.File; import java.io.IOException; import org.apache.lucene.analysis.Analyzer; import org.apache.lucene.analysis.standard.StandardAnalyzer; import org.apache.lucene.index.IndexWriter; import org.apache.lucene.index.IndexWriterConfig; import org.apache.lucene.store.Directory; import org.apache.lucene.store.FSDirectory; import org.apache.lucene.util.Version; public class TestLock { private Directory dir; public static TestLock ttt; public static IndexWriter writer2; private Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_48); private IndexWriterConfig iwc = new IndexWriterConfig(Version.LUCENE_48, analyzer); public void init() throws Exception { String pathFile = "D://luceneindex"; try{ dir = FSDirectory. open(new File(pathFile)); } catch (IOException e) { throw new RuntimeException(e); } IndexWriter writer = getWriter(); System. out.println("init ok,test IndexWriter lock" ); LockTest( writer); //writer.close(); } public IndexWriter getWriter() throws Exception { //analyzer.close(); return new IndexWriter(dir, iwc); } public void LockTest(IndexWriter w1) { try{ if(w1.isLocked (dir )){ System. out.println("write1 locked" ); IndexWriterConfig iwc1 = new IndexWriterConfig(Version.LUCENE_48 , analyzer ); writer2 = new IndexWriter(dir, iwc1); } else{ System. out.println("write1 not locked" ); } } catch(Exception e){ e.printStackTrace(); } } public static void main(String[] args){ ttt = new TestLock(); try{ ttt.init(); } catch(Exception e){ e. printStackTrace(); } } }
报错信息:
org.apache.lucene.store.LockObtainFailedException : Lock obtain timed out: NativeFSLock@D:\luceneindex\write.lock: java.nio.channels.OverlappingFileLockException at org.apache.lucene.store.Lock.obtain( Lock.java:89) at org.apache.lucene.index.IndexWriter.<init>( IndexWriter.java:710) at TestLock.LockTest( TestLock.java:38) at TestLock.init( TestLock.java:26) at TestLock.main( TestLock.java:51) Caused by: java.nio.channels.OverlappingFileLockException at sun.nio.ch.SharedFileLockTable.checkList(Unknown Source) at sun.nio.ch.SharedFileLockTable.add(Unknown Source) at sun.nio.ch.FileChannelImpl.tryLock(Unknown Source) at java.nio.channels.FileChannel.tryLock(Unknown Source) at org.apache.lucene.store.NativeFSLock.obtain(NativeFSLockFactory.java:148) at org.apache.lucene.store.Lock.obtain( Lock.java:100) ... 4 more
从错误信息可以看到是IndexWriter获取锁出错导致,从堆栈信息可以看到,是在运行IndexWriter的构造方法时,涉及到锁的操作。
查看IndexWriter的相关源码:
doc: Opening an IndexWriter creates a lock file for the directory in use. Trying to open another IndexWriter on the same directory will lead to a LockObtainFailedException. The LockObtainFailedException is also thrown if an IndexReader on the same directory is used to delete documents from the index.
两个和锁相关的属性:
public static final String WRITE_LOCK_NAME = "write.lock" ; private Lock writeLock;
在IndexWriter的构造函数中:
public IndexWriter(Directory d, IndexWriterConfig conf) throws IOException .... directory = d; ..... writeLock = directory.makeLock(WRITE_LOCK_NAME); if (! writeLock.obtain(config .getWriteLockTimeout())) // obtain write lock(尝试获取锁) throw new LockObtainFailedException("Index locked for write: " + writeLock );
其中Lock是一个抽象类(org.apache.lucene.store.Lock):
锁的获取主要实现是在obtain方法:
public static long LOCK_POLL_INTERVAL = 1000; public static final long LOCK_OBTAIN_WAIT_FOREVER = -1; public final boolean obtain(long lockWaitTimeout) throws IOException { failureReason = null; boolean locked = obtain(); if (lockWaitTimeout < 0 && lockWaitTimeout != LOCK_OBTAIN_WAIT_FOREVER ) throw new IllegalArgumentException("lockWaitTimeout should be LOCK_OBTAIN_WAIT_FOREVER or a non-negative number (got " + lockWaitTimeout + ")"); long maxSleepCount = lockWaitTimeout / LOCK_POLL_INTERVAL; long sleepCount = 0; while (!locked) { if (lockWaitTimeout != LOCK_OBTAIN_WAIT_FOREVER && sleepCount++ >= maxSleepCount) { String reason = "Lock obtain timed out: " + this .toString(); if (failureReason != null) { reason += ": " + failureReason ; } LockObtainFailedException e = new LockObtainFailedException(reason); if (failureReason != null) { e.initCause( failureReason); } throw e; } try { Thread. sleep(LOCK_POLL_INTERVAL); } catch (InterruptedException ie) { throw new ThreadInterruptedException(ie); } locked = obtain(); } return locked; }
可以看到有由下面几个因素决定:
1.lockWaitTimeout 超时时间(总时间,超过这个时间即timeout),默认1000ms
2.LOCK_OBTAIN_WAIT_FOREVER 是否无限获取(-1),如果设置为-1,会永不超生
3.LOCK_POLL_INTERVAL 重试间隔时间,默认1000ms
在关闭IndexWriter时调用close方法(实现了Closeable接口的类都会有close方法)即可正常释放锁
更改代码为如下即可:
public void LockTest(IndexWriter w1) { try{ if(w1.isLocked (dir )){ System. out.println("write1 locked" ); w1.close(); IndexWriterConfig iwc1 = new IndexWriterConfig(Version.LUCENE_48 , analyzer ); writer2 = new IndexWriter(dir, iwc1); } else{ System. out.println("write1 not locked" ); } } catch(Exception e){ e.printStackTrace(); } }
另外关于lock的判断和unlock方法如下:
public static boolean isLocked(Directory directory) throws IOException { // 判断写目录是否被lock return directory.makeLock( WRITE_LOCK_NAME).isLocked(); } /** * Forcibly unlocks the index in the named directory. * <P> * Caution: this should only be used by failure recovery code, * when it is known that no other process nor thread is in fact * currently accessing this index. */ public static void unlock(Directory directory) throws IOException { // 解锁 directory.makeLock(IndexWriter. WRITE_LOCK_NAME).close(); }
本文出自 “菜光光的博客” 博客,请务必保留此出处http://caiguangguang.blog.51cto.com/1652935/1432795
lucene indexwriter锁机制,布布扣,bubuko.com
原文地址:http://caiguangguang.blog.51cto.com/1652935/1432795