分享一个数学计算

前几天群里不认识的一个哥们,需要计算一下:
A+B=C,其中 C 不包含 A,B 中任意数字的集合
一时闲来无事,正好练练手,用python写了下,代码如下:

# -*- coding: utf-8 -*-
import time

'''
A+B=C,C不包含A,B中任意数字
'''

'''按位切分数字,返回字典{0-9,0-1}
ex{
  0:1,
  9:1
} 说明结果中包含0,9两个数字
'''
def spiltNumber( num ):
    n = num
    result = {}
    while (n > 0):
        if(n % 10 not in result):
            result[n % 10] = 1
        n = n / 10
    return result

# 2个字典取并集,为空说明不重复
def checkDictContains(dict1,dict2):
    return len(dict.fromkeys([x for x in dict1 if x in dict2])) == 0

def getList(start, end):
    list = range(start, end)
    return list

def getResult(start,end):
    # 返回的结果列表
    result = []
    # 迭代的次数
    z = 0
    for i in range(start, end):
        print("执行矩阵运算 x:%d,y:(%d~%d)..." % (i,i,end))
        for j in range(i, end):
            z += 1
            # 将左侧乘数转成倒排索引
            leftNumberDict = dict(spiltNumber(i), **spiltNumber(j))
            # 结果转换成倒排索引
            count = i + j
            countDict = spiltNumber(count)
            if (checkDictContains(countDict, leftNumberDict)):
                result.append(" %d + %d = %d" % (i, j, count))
                print("匹配结果: %d + %d = %d" % (i, j, count))
    print("匹配的结果数:%d" % (len(result)))
    print("迭代总次数:%d" % (z))

if __name__ == '__main__':
    startTime = time.time()
    getResult(100,999)
    print("执行时间:%d s" % (time.time()-startTime))

执行结果如下:

...
执行矩阵运算 x:992,y:(992~999)...
执行矩阵运算 x:993,y:(993~999)...
执行矩阵运算 x:994,y:(994~999)...
执行矩阵运算 x:995,y:(995~999)...
执行矩阵运算 x:996,y:(996~999)...
执行矩阵运算 x:997,y:(997~999)...
执行矩阵运算 x:998,y:(998~999)...
匹配的结果数:55509
迭代总次数:404550
执行时间:4 s

随后,这哥们需要计算A,B都为6位数,即100000<A<999999,100000<B<999999,
哎呀,这个数据量就比较大了,简单分析了下,大概计算量是10E12/2次:

这么大的数据量,靠单机运算,速度肯定就慢了,正好手上有集群资源,又学习了storm,于是拿这个需求练练手。(ps:其实这个场景更适合spark的RDD,目前还没学完,后面补上)
storm ui如下:

执行结果:

2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999885已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999903已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999870已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999936已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999970已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999990已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999985已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999915已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999910已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999928已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999780已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999800已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999798已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999796已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999820已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999836已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.553 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999930已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.554 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999927已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.554 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999923已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.554 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999855已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.554 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999858已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.554 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999857已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.554 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999882已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.554 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999937已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.554 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999967已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.554 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999974已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.554 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999971已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.554 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999912已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.554 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999931已获取到0条匹配结果,总计623112451条结果
2019-01-06 12:52:59.554 c.d.b.s.b.CountBlot Thread-9-CounterBolt-executor[1 1] [INFO] x:999925已获取到0条匹配结果,总计623112451条结果

代码已经上传github,有兴趣的小伙伴自行查看:
https://github.com/qiandaxian/matrix.git
下的sotrm文件夹

2019-1-9更新

重点来啦 spark scala的实现

源码如下:

package com.daxian
import org.apache.spark.SparkConf
import org.apache.spark.sql.SparkSession

/**
  * @author qiandaxian
  */
object SparkMatrix {
  /**
    * 判断x+y是否满足 x+y=C,C不包含x,y两数中任意数字
    */
  def checkResult(x: Int,y: Int): Boolean ={
    return (
        ( x.toString.toCharArray.toSet | y.toString.toCharArray.toSet)
        & (x + y).toString.toCharArray.toSet
      ).size == 0
  }

  def main(args: Array[String]) {
    val startNumber = args(0).toInt
    val endNumber = args(1).toInt

//  startNumber = 100
//  endNumber = 999

    //x轴数据集合
    var x = startNumber to endNumber

    val conf = new SparkConf()
        //本地模式
        //.setMaster("local").setAppName("matrix-count")

    val spark = SparkSession
                  .builder.config(conf)
                  .getOrCreate()

    val count = spark.sparkContext.parallelize(x,endNumber).map{x =>
      //y轴数据集合
      (x to endNumber)
          .filter(a => checkResult(x,a))
          .length
    }.reduce(_ + _)

    println("count is "+ count)
    spark.stop()
  }
}

有木有感觉草鸡简单。
执行日志:

2019-01-09 18:14:45 INFO  TaskSetManager:54 - Starting task 15857.0 in stage 0.0 (TID 15857, 192.168.207.178, executor 1, partition 15857, PROCESS_LOCAL, 7870 bytes)
2019-01-09 18:14:45 INFO  TaskSetManager:54 - Finished task 15829.0 in stage 0.0 (TID 15829) in 4600 ms on 192.168.207.178 (executor 1) (15834/999999)
2019-01-09 18:14:45 INFO  TaskSetManager:54 - Starting task 15858.0 in stage 0.0 (TID 15858, 192.168.207.178, executor 1, partition 15858, PROCESS_LOCAL, 7870 bytes)
2019-01-09 18:14:45 INFO  TaskSetManager:54 - Finished task 15834.0 in stage 0.0 (TID 15834) in 4378 ms on 192.168.207.178 (executor 1) (15835/999999)
2019-01-09 18:14:45 INFO  TaskSetManager:54 - Starting task 15859.0 in stage 0.0 (TID 15859, 192.168.207.178, executor 1, partition 15859, PROCESS_LOCAL, 7870 bytes)
2019-01-09 18:14:45 INFO  TaskSetManager:54 - Finished task 15832.0 in stage 0.0 (TID 15832) in 4545 ms on 192.168.207.178 (executor 1) (15836/999999)
2019-01-09 18:14:45 INFO  TaskSetManager:54 - Starting task 15860.0 in stage 0.0 (TID 15860, 192.168.207.177, executor 0, partition 15860, PROCESS_LOCAL, 7870 bytes)
2019-01-09 18:14:45 INFO  TaskSetManager:54 - Finished task 15835.0 in stage 0.0 (TID 15835) in 4421 ms on 192.168.207.177 (executor 0) (15837/999999)

ui页面:

发表评论

电子邮件地址不会被公开。 必填项已用*标注