Drawing in multiple UIViews causes lag. Why?

Question

I'm making a multiplayer game which involves drawing lines. I have 10 different custom UIViews where I draw an line using Core Graphics in an unique color. All views are stacked upon each other and are semi-transparent. The reason why I draw each color in corresponding UIView and not a single one is because I want to let the user be able to clear a specific view. However, If I draw in all of these views the same time, significant amount of lag will occur. If I check the time profiler most of the time is spent on drawing. But it's not that bad and should not cause that much lag. So, is there any way to let me draw in multiple UIViews (or something else), without causing this lag. Is it my code who causes this lag or is it the UIView?

- (void)drawRect:(CGRect)rect {
//create a image reference from the bitmap and then draw it in the context of the UIView. 

CGContextRef context = UIGraphicsGetCurrentContext();

CGImageRef cacheImage = CGBitmapContextCreateImage(cacheContext);

  CGContextDrawImage(context, self.bounds, cacheImage);
   CGImageRelease(cacheImage);
}

-(void)draw{
 //drawing code. Could ignore the Beizer curve algorithm here.

 CGContextSetStrokeColorWithColor(cacheContext, [color CGColor]);
 CGContextSetFillColorWithColor(cacheContext, [[UIColor blueColor] CGColor]);

 CGContextSetLineCap(cacheContext, kCGLineCapButt);
 CGContextSetLineWidth(cacheContext, 6+thickness);

        point0 = point1;
        point1 = point2; // previous previous point
        point2 = pos; // previous touch point
        point3 = currentpos;

        double x0 = (point0.x > -1) ? point0.x : point1.x; //after 4 touches we should have a back anchor point, if not, use the current anchor point
        double y0 = (point0.y > -1) ? point0.y : point1.y; //after 4 touches we should have a back anchor point, if not, use the current anchor point
        double x1 = point1.x;
        double  y1 =  point1.y;
        double x2 =  point2.x;
        double  y2 =  point2.y;
        double  x3 = point3.x;
        double  y3 = point3.y;



    // Assume we need to calculate the control
    // points between (x1,y1) and (x2,y2).
    // Then x0,y0 - the previous vertex,
    //      x3,y3 - the next one.

    double xc1 = (x0 + x1) / 2.0;
    double yc1 = (y0 + y1) / 2.0;
    double xc2 = (x1 + x2) / 2.0;
    double yc2 = (y1 + y2) / 2.0;
    double xc3 = (x2 + x3) / 2.0;
    double yc3 = (y2 + y3) / 2.0;

    double len1 = sqrt((x1-x0) * (x1-x0) + (y1-y0) * (y1-y0));
    double len2 = sqrt((x2-x1) * (x2-x1) + (y2-y1) * (y2-y1));
    double len3 = sqrt((x3-x2) * (x3-x2) + (y3-y2) * (y3-y2));

    double k1 = len1 / (len1 + len2);
    double k2 = len2 / (len2 + len3);

    double xm1 = xc1 + (xc2 - xc1) * k1;
    double ym1 = yc1 + (yc2 - yc1) * k1;

    double xm2 = xc2 + (xc3 - xc2) * k2;
    double ym2 = yc2 + (yc3 - yc2) * k2;
    double smooth_value = 0.8;
    // Resulting control points. Here smooth_value is mentioned
    // above coefficient K whose value should be in range [0...1].
    float ctrl1_x = xm1 + (xc2 - xm1) * smooth_value + x1 - xm1;
    float ctrl1_y = ym1 + (yc2 - ym1) * smooth_value + y1 - ym1;

    float ctrl2_x = xm2 + (xc2 - xm2) * smooth_value + x2 - xm2;
    float ctrl2_y = ym2 + (yc2 - ym2) * smooth_value + y2 - ym2;



        CGContextMoveToPoint(cacheContext, point1.x, point1.y);
        CGContextAddCurveToPoint(cacheContext, ctrl1_x, ctrl1_y, ctrl2_x, ctrl2_y, point2.x, point2.y);
        CGContextStrokePath(cacheContext);




    CGRect dirtyPoint1 = CGRectMake(point1.x-(8+thickness)/2, point1.y-(8+thickness)/2, (8+thickness), (8+thickness));
    CGRect dirtyPoint2 = CGRectMake(point2.x-(8+thickness)/2, point2.y-(8+thickness)/2, (8+thickness), (8+thickness));

    [self setNeedsDisplayInRect:CGRectUnion(dirtyPoint1, dirtyPoint2)];

}