[英]Items set with spymemcached cannot be fetched with php memcached
我正在使用spymemcached。 我設置了幾個項目。 然后我運行一個PHP腳本,然而我無法使用php memcached獲取所有這些項目。 PHP-Memcached只能部分檢索這些項目。
我無法改變php的哈希算法或分發策略。 在我們的系統中,我們使用默認散列(根據php.net文檔,這是jenkin的一次一個)。 並且分發策略是php-memcached的模數。 我已經讀過spymemcached使用一致性散列。 有什么方法可以在spymemcached中使用模數散列。
換句話說,如何使spymemcached的set操作或任何其他存儲操作與php-memcached的get操作兼容?
如果spymemcached無法做到這一點,java中是否有任何其他memcached客戶端允許我這樣做?
幫助不僅會被欣賞,也會得到賞金。
Java代碼:
public static void main(String [] args) {
List<InetSocketAddress> addrs = new ArrayList<>();
addrs.add(new InetSocketAddress("10.90.12.87", 11211));
addrs.add(new InetSocketAddress("10.90.12.87", 11311));
try {
MemcachedClient memcache = new MemcachedClient(addrs);
memcache.add("foo", 0, "bar");
memcache.add("sample", 0, "key");
memcache.add("try", 0, "another");
memcache.add("ax-spadg-list", 0, "3045,6645");
} catch (IOException ex) {
Logger.getLogger(CategoryDataOperator.class.getName()).log(Level.SEVERE, null, ex);
}
System.out.println("Done");
}
PHP代碼:
<?php
$mem = new Memcached();
$mem->addServer('10.90.12.87', 11211);
$mem->addServer('10.90.12.87', 11311);
var_dump $mem->get('foo');
var_dump($mem->get('try'));
var_dump($mem->get('sample'));
var_dump($mem->get('ax-spadg-list'));
問題是關於Hash,默認的php-memcached哈希是
(Jenkins one-at-a-time)項密鑰散列算法
而spymemcached哈希列表是:
NATIVE_HASH
:簡單的Native hash (String.hashCode()).
與默認的php-memcached Memcached::HASH_DEFAULT
不匹配 CRC_HASH
=> Memcached::HASH_CRC
FNV1_64_HASH
=> Memcached::HASH_FNV1_64
FNV1A_64_HASH
=> Memcached::HASH_FNV1A_64
FNV1_32_HASH
=> Memcached::HASH_FNV1_32
FNV1A_32_HASH
=> Memcached::HASH_FNV1A_32
KETAMA_HASH
=>“ketama使用的基於MD5的哈希算法。” 也許Memcached::HASH_MD5
但無論如何不是Memcached::HASH_DEFAULT
因此,如果您無法更改PHP客戶端配置或擴展spymemcached lib,則兩個庫之間不會直接匹配。
解決方案1:如果您查看歷史記錄 (您可以使用php客戶端哈希修改示例)。
解決方案2:否則你可以創建一個JenkinHash類(我復制過Xmemcached代碼: https : //code.google.com/p/xmemcached/source/browse/trunk/src/main/java/net/rubyeye/xmemcached/ HashAlgorithm.java?r=801#176 [但考慮到Xmemcached許可證並將作者/許可證保留在源代碼中])
import net.spy.memcached.HashAlgorithm;
import java.io.UnsupportedEncodingException;
public class JenkinsHash implements HashAlgorithm {
@Override
public long hash(String k) {
try {
int hash = 0;
for (byte bt : k.getBytes("utf-8")) {
hash += (bt & 0xFF);
hash += (hash << 10);
hash ^= (hash >>> 6);
}
hash += (hash << 3);
hash ^= (hash >>> 11);
hash += (hash << 15);
return hash;
} catch (UnsupportedEncodingException e) {
throw new IllegalStateException("Hash function error", e);
}
}
}
然后:
import net.spy.memcached.*;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.util.ArrayList;
import java.util.List;
import java.util.logging.Level;
import java.util.logging.Logger;
public class Main {
public static void main(String[] args) throws IOException {
List<InetSocketAddress> addrs = new ArrayList<InetSocketAddress>();
addrs.add(new InetSocketAddress("127.0.0.1", 11211));
addrs.add(new InetSocketAddress("172.28.29.22", 11211));
try {
ConnectionFactory connectionFactory = new ConnectionFactoryBuilder()
.setProtocol(ConnectionFactoryBuilder.Protocol.TEXT)
.setHashAlg(new JenkinsHash())
.setLocatorType(ConnectionFactoryBuilder.Locator.ARRAY_MOD).build();
MemcachedClient memcache = new MemcachedClient(connectionFactory, addrs);
memcache.add("foo", 0, "bar2");
memcache.add("sample", 0, "key");
memcache.add("try", 0, "another");
memcache.add("ax-spadg-list", 0, "3045,6645");
} catch (IOException ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
}
System.out.println("Done");
}
}
使用php腳本:
<?php
$memcached = new Memcached();
$memcached->addserver('127.0.0.1', 11211);
$memcached->addserver('172.28.29.22', 11211);
var_dump($memcached->get('foo'));
var_dump($memcached->get('try'));
var_dump($memcached->get('sample'));
var_dump($memcached->get('ax-spadg-list'));
測試:
$ echo "flush_all" | nc 172.28.29.22 11211 && echo "flush_all" | nc 127.0.0.1 11211
OK
OK
$ php mem.php
bool(false)
bool(false)
bool(false)
bool(false)
RUN JAVA
$ php mem.php
string(4) "bar2"
string(7) "another"
string(3) "key"
string(9) "3045,6645"
解決方案3:使用https://code.google.com/p/xmemcached/和ONE_AT_A_TIME
哈希算法
import net.rubyeye.xmemcached.HashAlgorithm;
import net.rubyeye.xmemcached.MemcachedClient;
import net.rubyeye.xmemcached.MemcachedClientBuilder;
import net.rubyeye.xmemcached.XMemcachedClientBuilder;
import net.rubyeye.xmemcached.exception.MemcachedException;
import net.rubyeye.xmemcached.impl.ArrayMemcachedSessionLocator;
import java.io.IOException;
import java.net.InetSocketAddress;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.TimeoutException;
public class Main {
public static void main(String[] args) throws IOException, InterruptedException, MemcachedException, TimeoutException {
List<InetSocketAddress> addrs = new ArrayList<InetSocketAddress>();
addrs.add(new InetSocketAddress("127.0.0.1", 11211));
addrs.add(new InetSocketAddress("172.28.29.22", 11211));
MemcachedClientBuilder builder = new XMemcachedClientBuilder(addrs);
builder.setSessionLocator(new ArrayMemcachedSessionLocator(HashAlgorithm.ONE_AT_A_TIME));
MemcachedClient memcachedClient = builder.build();
memcachedClient.set("foo", 0, "bar2");
memcachedClient.set("sample", 0, "key");
memcachedClient.set("try", 0, "another");
memcachedClient.set("ax-spadg-list", 0, "3045,6645");
memcachedClient.shutdown();
System.out.println("Done");
}
}
spymemcached支持的哈希算法在這里: https : //github.com/couchbase/spymemcached/blob/master/src/main/java/net/spy/memcached/DefaultHashAlgorithm.java
您應該能夠通過使用ConnectionFactory來創建MemcachedClient來更改哈希算法。 做這樣的事情:
ConnectionFactoryBuilder builder = new ConnectionFactoryBuilder();
builder.setHashAlgorithm(HashAlgorithm.CRC_HASH);
ConnectionFactory factory = builder.build();
MemcachedClient client = new MemcachedClient(Arrays.asList(new InetSocketAddr("localhost", 11211)), factory);
回復:Kakawait(以及Shades88)
解決方案#2不正確,因為xmemcached沒有正確移植使用unsigned的Jenkins哈希的原始C代碼。 修復此問題還將解決Shades88看到的ArrayIndexOutOfBoundsException。
public class JenkinsHash implements HashAlgorithm {
@Override
public long hash(String k) {
try {
int hash = 0;
for (byte bt : k.getBytes("utf-8")) {
hash += (bt & 0xFF);
hash += (hash << 10);
hash ^= (hash >>> 6);
}
hash += (hash << 3);
hash ^= (hash >>> 11);
hash += (hash << 15);
// the hash variable in the original C code is a uint32.
// convert the java signed int to an "unsigned",
// represented via a long:
return hash & 0xFFFFFFFFl;
} catch (UnsupportedEncodingException e) {
throw new IllegalStateException("Hash function error", e);
}
}
}
// Unit test
public class JenkinsHashTest {
@Test
public void testHash() throws Exception {
JenkinsHash j = new JenkinsHash();
Properties p = new Properties();
// This file contains k/v mappings,
// with values generated by the reference C code
p.load(new FileReader("src/test/resources/jenkinsHashTest.dat"));
for (Entry<Object, Object> entry : p.entrySet()) {
long result = j.hash((String)entry.getKey());
// Print out hash mismatches
if (result != Long.parseLong((String)entry.getValue())) {
System.out.println("Key: " + (String)entry.getKey());
System.out.println("Expected Hash Value: " + Long.parseLong((String)entry.getValue()));
System.out.println("Actual Hash Value: " + result);
}
assertEquals(result, Long.parseLong((String)entry.getValue()));
}
}
}
測試數據文件用於將Java代碼與C代碼進行比較。 構建C代碼然后散列一堆隨機單詞並將它們映射到一個文件中,如下所示:
jenkinsHashTest.dat:
sausage=2834523395
blubber=1103975961
pencil=3318404908
cloud=670342857
moon=2385442906
water=3403519606
computer=2375101981
school=1513618861
network=2981967937
hammer=1218821080
...添加任意多個
聲明:本站的技術帖子網頁,遵循CC BY-SA 4.0協議,如果您需要轉載,請注明本站網址或者原文地址。任何問題請咨詢:yoyou2525@163.com.